Extractor:

pipelinewise-tap-mysql

Loader:

pipelinewise-target-snowflake

Tables replicated via

LOG_BASED

replication method. It worked on my lower env, and same setting seem not to be working on higher env. Log on higher env: Also tried with a --full-refresh run, same issue. Job is completed, but no tables were synced.

root@data-pipes-hos-data-5595c49d87-sfnpb:/project# meltano schedule run hos-data --full-refresh
2022-06-24T10:36:00.511788Z [info     ] Running extract & load...      job_id=hos-data name=meltano run_id=a196b154-67d9-bfcb-88f9dbb5c10b
2022-06-24T10:36:08.778653Z [info     ] Performing full refresh, ignoring state left behind by any previous runs.
2022-06-24T10:36:14.799111Z [info     ] time=2022-06-24 10:36:14 name=tap_mysql level=INFO message=Server Parameters: version: 5.7.12-log, wait_timeout: 28800, innodb_lock_wait_timeout: 50, max_allowed_packet: 4194304, interactive_timeout: 28800 cmd_type=extractor job_id=hos-data name=hos_db run_id=a196b154-67d9-4bfe-bfcb-88f9dbb5c10b stdio=stderr
2022-06-24T10:36:14.799925Z [info     ] time=2022-06-24 10:36:14 name=tap_mysql level=INFO message=Server SSL Parameters(blank means SSL is not active): [ssl_version: ], [ssl_cipher: ] cmd_type=extractor job_id=hos-data name=hos_db run_id=a196b154-67d9-bfcb-88f9dbb5c10b stdio=stderr
2022-06-24T10:36:15.597816Z [info     ] time=2022-06-24 10:36:15 name=target_snowflake level=INFO message=Getting catalog objects from table cache... cmd_type=loader job_id=hos-data name=data-warehouse run_id=a196b154-67d9-bfcb-88f9dbb5c10b stdio=stderr
2022-06-24T10:36:19.905533Z [info     ] time=2022-06-24 10:36:19 name=target_snowflake level=INFO message=Emitting state {"currently_syncing": null} cmd_type=loader job_id=hos-data name=data-warehouse run_id=a196b154-67d9-bfcb-88f9dbb5c10b stdio=stderr
2022-06-24T10:36:19.914749Z [info     ] Incremental state has been updated at 2022-06-24 10:36:19.914529.
2022-06-24T10:36:20.117680Z [info     ] Extract & load complete!       job_id=hos-data name=meltano run_id=a196b154-67d9-bfcb-88f9dbb5c10b
2022-06-24T10:36:20.118059Z [info     ] Transformation skipped.        job_id=hos-data name=meltano run_id=a196b154-

Any help is highly appreciated.

Usually this problem is caused by select rules. In this case I wonder if maybe the schema name is different between the two environments? That would make the stream names different which would mean you'll need different

select

rules.

yes that is correct. The db name is different. I now see the issue. Thanks 🙂 i only changes the db name in env variables, but not in select section in meltano.yml