yijie_wang
06/24/2022, 2:56 PMmeltano elt tap-snowflake target-jsonl
I follow all the steps from this thread(https://meltano.slack.com/archives/C01TCRBBJD7/p1621519111034600), but it will return any record
- name: tap-snowflake
variant: transferwise
pip_url: pipelinewise-tap-snowflake
capabilities:
- catalog
- discover
config:
account: test
dbname: TEST1
role: test
user: test
warehouse: test
password: $SNOWFLAKE_PASSWORD
tables: TEST1.AB_TESTING.RELATED_PARTY_OPPORTUNITY
# catalog: extract/tap-snowflake-related-party.properties.json
metadata:
TEST1-AB_TESTING-RELATED_PARTY_OPPORTUNITY:
replication-method: FULL_TABLE
select:
- TEST1.AB_TESTING.RELATED_PARTY_OPPORTUNITY*.*
- TEST1-AB_TESTING-RELATED_PARTY_OPPORTUNITY*.*
- RELATED_PARTY_OPPORTUNITY*.*
Below is the output. I add the LOGG.info in the get_stream function which return resovle_catelog result {‘streams’: []}
2022-06-24T14:26:18.150996Z [info ] time=2022-06-24 14:26:18 name=tap_snowflake level=INFO message=resovle_catelog result {'streams': []} cmd_type=extractor name=tap-snowflake run_id=1c2d76e8-9a1c-4c37-b336-c935477e6c87 state_id=2022-06-24T142611--tap-snowflake--target-jsonl stdio=stderr
2022-06-24T14:26:18.151827Z [debug ] {"type": "STATE", "value": {"currently_syncing": null}} cmd_type=extractor name=tap-snowflake (out) run_id=1c2d76e8-9a1c-4c37-b336-c935477e6c87 state_id=2022-06-24T142611--tap-snowflake--target-jsonl stdio=stdout
2022-06-24T14:26:18.315641Z [debug ] {"currently_syncing": null} cmd_type=loader name=target-jsonl (out) run_id=1c2d76e8-9a1c-4c37-b336-c935477e6c87 state_id=2022-06-24T142611--tap-snowflake--target-jsonl stdio=stdout
2022-06-24T14:26:18.339183Z [info ] Incremental state has been updated at 2022-06-24 14:26:18.339055.
2022-06-24T14:26:18.340450Z [debug ] Incremental state: {'currently_syncing': None}
2022-06-24T14:26:18.345291Z [debug ] Deleted configuration at /project/.meltano/run/elt/2022-06-24T142611--tap-snowflake--target-jsonl/1c2d76e8-9a1c-4c37-b336-c935477e6c87/target.c6523ca2-3ebe-464b-84d0-2fd6f79740ac.config.json
2022-06-24T14:26:18.347285Z [debug ] Deleted configuration at /project/.meltano/run/elt/2022-06-24T142611--tap-snowflake--target-jsonl/1c2d76e8-9a1c-4c37-b336-c935477e6c87/tap.fa14a899-fb00-4797-96d9-79e7933f41e2.config.json
2022-06-24T14:26:18.348266Z [info ] Extract & load complete! name=meltano run_id=1c2d76e8-9a1c-4c37-b336-c935477e6c87 state_id=2022-06-24T142611--tap-snowflake--target-jsonl
2022-06-24T14:26:18.349153Z [info ] Transformation skipped. name=meltano run_id=1c2d76e8-9a1c-4c37-b336-c935477e6c87 state_id=2022-06-24T142611--tap-snowflake--target-jsonlpat_nadolny
06/24/2022, 3:33 PM* for your metadata selection on one of the tables and see if that gets some data flowing. Also maybe the first * as part of your select criteria TEST1.AB_TESTING.RELATED_PARTY_OPPORTUNITY*.* -> TEST1.AB_TESTING.RELATED_PARTY_OPPORTUNITY.* . I dont know but taking a few guesses at things I'd try 😄yijie_wang
06/24/2022, 4:35 PMselect:
- TEST1-AB_TESTING-RELATED_PARTY_OPPORTUNITY.*yijie_wang
06/24/2022, 4:36 PMselect:
- <"tap_stream_id">-<table_name>.*