Hello, I'm having trouble with using the extractor tap-mysql (transferwise variant),i have the following error :

2023-02-16T084031.322035Z [info ] time=2023-02-16 084031 name=tap_mysql level=CRITICAL message=unknown encoding: utf8mb3 cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.322250Z [info ] Traceback (most recent call last): cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.323867Z [info ] File "/projects/.meltano/extractors/tap-mysql/venv/bin/tap-mysql", line 8, in <module> cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.324317Z [info ] sys.exit(main()) cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.324561Z [info ] File "/projects/.meltano/extractors/tap-mysql/venv/lib/python3.9/site-packages/tap_mysql/__init__.py", line 443, in main cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.324819Z [info ] raise exc cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.325027Z [info ] File "/projects/.meltano/extractors/tap-mysql/venv/lib/python3.9/site-packages/tap_mysql/__init__.py", line 440, in main cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.325232Z [info ] main_impl() cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.325417Z [info ] File "/projects/.meltano/extractors/tap-mysql/venv/lib/python3.9/site-packages/tap_mysql/__init__.py", line 429, in main_impl cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.325641Z [info ] do_sync(mysql_conn, args.config, args.catalog, state) cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.325854Z [info ] File "/projects/.meltano/extractors/tap-mysql/venv/lib/python3.9/site-packages/tap_mysql/__init__.py", line 385, in do_sync cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.326045Z [info ] sync_binlog_streams(mysql_conn, binlog_catalog, config, state) cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

2023-02-16T084031.326215Z [info ] File "/projects/.meltano/extractors/tap-mysql/venv/lib/python3.9/site-packages/tap_mysql/__init__.py", line 368, in sync_binlog_streams cmd_type=elb consumer=False name=tap-mysql producer=True stdio=stderr string_id=tap-mysql

Do you know how to fix the issue or using a specific configuration parameters?

I think this is an issue with default client character set. You can use the session_sqls parameter to set session options when the tap connects; I think for this you probably want character_set_connection or something like that.

Thank you for the answer, i already tried that, but we are using a new version of MariaDB that only allow utf8mb3 or utf8mb4

I'm having a hard time reconciling that. How can utf8mb3 be an unknown encoding the database only allows utf8mb3 or utf8mb3?

MariaDB > 10.6 has only utf8mb3 and utf8mb4, utf8 has been totally removed. But it's seems that the tap-mysql extractor doesn't support utf8mb3 or utf8mb4 regarding the stacktrace that i have ("unknown encoding: utf8mb3")

I would bet that the issue is the underlying library the tap uses to communicate with MySQL... which I think is pymysql? Maybe try building the tap from source with a later version of that and see if it helps?

Yes i think it's pymysql that cause the issue I found a workaround is to update my data schema from utf8mb3 to utf8mb4 and the extractor is working fine

You are welcome. 🙂 Glad you got it working!