Hello I’m running mysql -> snowflake pipeline a...
# plugins-general
r
Hello I’m running mysql -> snowflake pipeline and it’s failing constantly on large tables (8 mln. rows). Had anyone run to this issue previously?
d
@ruslan_yangazitov Which tap-mysql are you using? I can't find the string
settimeout
in the two I know: https://github.com/transferwise/pipelinewise-tap-mysql/search?q=settimeout or https://github.com/singer-io/tap-mysql/search?q=settimeout
r
@douwe_maan I’m using singer-io variant
and transferwise variant for target-snowflake
d
Do you see a full stacktrace further up in the logs? The first step will be to find the code causing the issue
r
```tap-mysql | CRITICAL 'NoneType' object has no attribute 'settimeout' tap-mysql | Traceback (most recent call last): tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/tap_mysql/sync_strategies/incremental.py", line 81, in sync_table tap-mysql | params) tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/tap_mysql/sync_strategies/common.py", line 255, in sync_query tap-mysql | row = cursor.fetchone() tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 469, in fetchone tap-mysql | row = self.read_next() tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 464, in read_next tap-mysql | return self._conv_row(self._result._read_rowdata_packet_unbuffered()) tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1160, in _read_rowdata_packet_unbuffered tap-mysql | packet = self.connection._read_packet() tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/pymysql/connections.py", line 674, in _read_packet tap-mysql | recv_data = self._read_bytes(bytes_to_read) tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/pymysql/connections.py", line 707, in _read_bytes tap-mysql | CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") tap-mysql | pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query') tap-mysql | tap-mysql | During handling of the above exception, another exception occurred: tap-mysql | tap-mysql | Traceback (most recent call last): tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/bin/tap-mysql", line 8, in <module> tap-mysql | sys.exit(main()) tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/singer/utils.py", line 229, in wrapped tap-mysql | return fnc(*args, **kwargs) tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/tap_mysql/__init__.py", line 732, in main tap-mysql | do_sync(mysql_conn, args.config, catalog, state) tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/tap_mysql/__init__.py", line 679, in do_sync tap-mysql | sync_non_binlog_streams(mysql_conn, non_binlog_catalog, config, state) tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/tap_mysql/__init__.py", line 654, in sync_non_binlog_streams tap-mysql | do_sync_incremental(mysql_conn, catalog_entry, state, columns, optional_limit) tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/tap_mysql/__init__.py", line 527, in do_sync_incremental tap-mysql | incremental.sync_table(mysql_conn, catalog_entry, state, columns) tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/tap_mysql/sync_strategies/incremental.py", line 83, in sync_table tap-mysql | iterate_limit = False tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 69, in exit tap-mysql | self.close() tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 441, in close tap-mysql | self._result._finish_unbuffered_query() tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1177, in _finish_unbuffered_query tap-mysql | packet = self.connection._read_packet() tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/pymysql/connections.py", line 657, in _read_packet tap-mysql | packet_header = self._read_bytes(4) tap-mysql | File "/project/.meltano/extractors/tap-mysql/venv/lib/python3.6/site-packages/pymysql/connections.py", line 688, in _read_bytes tap-mysql | …
@douwe_maan sorry, my bad) Now I see the reason
Copy code
tap-mysql | CR.CR_SERVER_LOST, "Lost connection to MySQL server during query")
tap-mysql | pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query')
We’ll check that with the ops team.
d
@ruslan_yangazitov I helped someone else with this issue a few months ago: https://meltano.slack.com/archives/CFG3C3C66/p1610570729210000?thread_ts=1610560265.206500&amp;cid=CFG3C3C66
That thread also has some theories for why the connection would be lost. Increasing the ELT buffer size may be a workaround, but the real solution would be to have the tap reconnect when it loses the connection
r
That’s really helpful, thank you!