https://linen.dev logo
#singer-tap-development
Title
# singer-tap-development
f

fred_reimer

11/05/2021, 7:14 PM
Probably something simple, but I'm getting:
Copy code
tap-athena      | extractor |   File ".../meltano/.meltano/extractors/tap-athena/venv/lib/python3.9/site-packages/singer_sdk/tap_base.py", line 514, in discover_streams
tap-athena      | extractor |     for catalog_entry in self.catalog_dict["streams"]:
tap-athena      | extractor | KeyError: 'streams'
when I do a elt run. However, I can do a meltano invoke tap-athena --discover and get the discovery catalog. I'm trying to test a change before I submit the PR, to make the AWS credentials optional so they can be picked up via standard locations (env, default profile, instance creds, etc). With debug logging I get:
Copy code
time=2021-11-05 15:09:19 name=tap-athena level=INFO message=Skipping parse of env var settings...
time=2021-11-05 15:09:19 name=tap-athena level=INFO message=Config validation passed with 0 errors and 0 warnings.
time=2021-11-05 15:09:19 name=root level=INFO message=Operator '__else__=None' was not found. Unmapped streams will be included in output.
time=2021-11-05 15:09:19 name=botocore.credentials level=INFO message=Found credentials in shared credentials file: ~/.aws/credentials
time=2021-11-05 15:09:50 name=botocore.credentials level=INFO message=Found credentials in shared credentials file: ~/.aws/credentials
[2021-11-05 15:10:21,414] [82571|MainThread|root] [DEBUG] Deleted configuration at .../meltano/.meltano/run/tap-athena/tap.....config.json
So it woks with discover, but when used as an actual target it can't use the discover to find the streams...