I have a supposedly trivial question: How can I ma...
# troubleshooting
a
I have a supposedly trivial question: How can I make Meltano to forget about the past altogether and start syncing as per the config anew (without reinstalling it, obviously)? I had a config that moved certain tables from a MySQL tap into a BigQuery target. Then I changed the config to sync other tables -- but one. That one, though, has a slightly modified structure. When I run the project it picks up the only table that used to be there, slightly warns about the structural changes, syncs it, but it will not do anything with the new tables. The log is quite lengthy but the gist is (I guess):
Copy code
2023-06-02T15:35:45.383673Z [info     ] Environment 'prod' is active
2023-06-02T15:35:53.664389Z [info     ] time=2023-06-02 15:35:53 name=tap_mysql level=INFO message=Server Parameters: version: 8.0.17, wait_timeout: 28800, innodb_lock_wait_timeout: 3600, max_allowed_packet: 67108864, interactive_timeout: 28800 cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql
2023-06-02T15:35:54.398324Z [info     ] time=2023-06-02 15:35:54 name=tap_mysql level=INFO message=Server SSL Parameters(blank means SSL is not active): [ssl_version: ], [ssl_cipher: ] cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql
2023-06-02T15:35:54.673738Z [info     ] time=2023-06-02 15:35:54 name=tap_mysql level=WARNING message=Columns {'<REDACTED>', '<REDACTED>', '<REDACTED>', '<REDACTED>', '<REDACTED>', '<REDACTED>', '<REDACTED>', '<REDACTED>'} were selected but do not exist. cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql
2023-06-02T15:35:54.727212Z [info     ] INFO Updating state with {'currently_syncing': 'reportdb-report_data'} cmd_type=elb consumer=True name=target-bigquery producer=False stdio=stderr string_id=target-bigquery
2023-06-02T15:35:54.728092Z [info     ] time=2023-06-02 15:35:54 name=tap_mysql level=INFO message=Beginning sync for view reportdb.report_data cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql
2023-06-02T15:35:54.728549Z [info     ] time=2023-06-02 15:35:54 name=tap_mysql level=INFO message=Stream reportdb-report_data is using full table replication cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

... [syncs records in reportdb-report_data only] ...

2023-06-02T15:37:20.769637Z [info     ] INFO Pushing state: {'currently_syncing': None, 'bookmarks': {'reportdb-report_data': {'initial_full_table_complete': True}}} cmd_type=elb consumer=True name=target-bigquery producer=False stdio=stderr string_id=target-bigquery
2023-06-02T15:37:20.894796Z [info     ] Incremental state has been updated at 2023-06-02 15:37:20.894457.
2023-06-02T15:37:21.119848Z [info     ] Block run completed.
The same happens if I run it with the
--full-refresh
option. Is it a caching issue of some sort? How could I reset it? What is the standard workflow in this case?
t
Yeah, the catalog is cached. Remove
.meltano/run/tap-mysql/tap.properties.cache_key
and then run meltano again.
a
It worked! thankyou