Hi, tap-mysql error on LOG_BASED replication-method.

time=2022-06-30 14:08:08 name=tap_mysql level=CRITICAL message=only INCREMENTAL, LOG_BASED, and FULL TABLE replication methods are supported
Traceback (most recent call last):

meltano_projects/test_meltano/.meltano/extractors/tap-mysql/venv/lib/python3.9/site-packages/tap_mysql/__init__.py", line 354, in sync_non_binlog_streams
    raise Exception("only INCREMENTAL, LOG_BASED, and FULL TABLE replication methods are supported")
Exception: only INCREMENTAL, LOG_BASED, and FULL TABLE replication methods are supported

I am testing this on local mysql 5.7.36 container. not able to figure out the issue.

- name: tap-mysql
        config:
          host: 127.0.0.1
          port: 3303
          user: *****
          database: hr
        select:
        - hr-jobs.job_id
        - hr-jobs.job_title
        - hr-jobs.max_salary
        - hr-jobs.min_salary
        - hr-locations.city
        - hr-locations.country_id
        - hr-locations.state_province
        metadata:
          hr-jobs:
            replication-method: LOG_BASED
          hr-locations:
            replication-method: LOG_BASED

I've spent some time trying to find a solution for this issue and this was the way i've found to make it work:

- name: tap-mysql
        config:
          metadata.*.test_table*.selected: true
          metadata.*.*.selected: true
          metadata.myDB-test_table.selected: true
          metadata.myDB-test_table.replication-method: LOG_BASED
        select:
        - test_table.*
        metadata:
          myDB-test_table:
            replication-method: LOG_BASED

Not sure why, but the replication that is inside the metadata tag was not working. Adding the metadata.myDB-test_table.replication-method: LOG_BASED inside the config tab solved the issue.

thank you, let me try it out

can you try to run it with debug enable and check if there's any error? You need to add --log-level=debug

I did check the debug mode and did not see any warnings or error. I can see it detecting the binlog by tap-mysql and target-snowflake mentioning about respective snowflake tables. Actually snowflake tables are not getting created.

2022-07-01T21:23:16.011590Z [info     ] time=2022-07-01 14:23:16 name=target_snowflake level=INFO message=Getting catalog objects from table cache... cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2022-07-01T21:23:19.403458Z [info     ] time=2022-07-01 14:23:19 name=target_snowflake level=INFO message=Table PUBLIC."JOBS" exists cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2022-07-01T21:23:21.217703Z [info     ] time=2022-07-01 14:23:21 name=target_snowflake level=INFO message=Table PUBLIC."LOCATIONS" exists cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2022-07-01T21:23:22.942747Z [info     ] time=2022-07-01 14:23:22 name=target_snowflake level=INFO message=Emitting state {"currently_syncing": null, "bookmarks": {"hr-jobs": {"version": 1656650608245, "log_file": "bin.000001", "log_pos": 1011}, "hr-locations": {"version": 1656650608291, "log_file": "bin.000001", "log_pos": 1011}}} cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2022-07-01T21:23:22.947892Z [info     ] Incremental state has been updated at 2022-07-01 21:23:22.947770.
2022-07-01T21:23:22.948103Z [debug    ] Incremental state: {'currently_syncing': None, 'bookmarks': {'hr-jobs': {'version': 1656650608245, 'log_file': 'bin.000001', 'log_pos': 1011}, 'hr-locations': {'version': 1656650608291, 'log_file': 'bin.000001', 'log_pos': 1011}}}
2022-07-01T21:23:22.948225Z [info     ] {"currently_syncing": null, "bookmarks": {"hr-jobs": {"version": 1656650608245, "log_file": "bin.000001", "log_pos": 1011}, "hr-locations": {"version": 1656650608291, "log_file": "bin.000001", "log_pos": 1011}}} cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stdout string_id=target-snowflake

Hi @arjunkumar_krishnankutty, sorry i was absent last week. Did you manage to sort your issue?