Hello Meltano, I am using tap-mysql and when running with batch config my command runs fine, however as this functionality doesn't allow audit fields I am removing the batch config from the configuration and it then stops working and now I am getting this error:

sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query')
(Background on this error at: <https://sqlalche.me/e/14/e3q8>)
Exception ignored in: <function SSCursor.close at 0x0000000000000000>
Traceback (most recent call last):
  File "path\\to\\your\\project\\.meltano\\extractors\\tap-mysql\\venv\\Lib\\site-packages\\pymysql\\cursors.py", line 434, in close
  File "path\\to\\your\\project\\.meltano\\extractors\\tap-mysql\\venv\\Lib\\site-packages\\pymysql\\connections.py", line 1310, in _finish_unbuffered_query
  File "path\\to\\your\\project\\.meltano\\extractors\\tap-mysql\\venv\\Lib\\site-packages\\pymysql\\connections.py", line 739, in _read_packet
  File "path\\to\\your\\project\\.meltano\\extractors\\tap-mysql\\venv\\Lib\\site-packages\\pymysql\\connections.py", line 776, in _read_bytes
AttributeError: 'NoneType' object has no attribute 'settimeout'
Exception ignored in: <function MySQLResult.__del__ at 0x0000000000000000>
Traceback (most recent call last):
  File "path\\to\\your\\project\\.meltano\\extractors\\tap-mysql\\venv\\Lib\\site-packages\\pymysql\\connections.py", line 1196, in __del__
  File "path\\to\\your\\project\\.meltano\\extractors\\tap-mysql\\venv\\Lib\\site-packages\\pymysql\\connections.py", line 1310, in _finish_unbuffered_query
  File "path\\to\\your\\project\\.meltano\\extractors\\tap-mysql\\venv\\Lib\\site-packages\\pymysql\\connections.py", line 739, in _read_packet
  File "path\\to\\your\\project\\.meltano\\extractors\\tap-mysql\\venv\\Lib\\site-packages\\pymysql\\connections.py", line 776, in _read_bytes
AttributeError: 'NoneType' object has no attribute 'settimeout'

I think it's https://github.com/transferwise/pipelinewise/issues/503

hi @Edgar Ramírez (Arch.dev) that issue is not resolved and it is for another variant. I am using meltano labs. The odd thing is without batch config the job runs fine

Ah, then I think the MeltanoLabs variant needs either a similar

session_sqls

setting or to hardcode those statements

@Edgar Ramírez (Arch.dev) I have those set too. Also there is a inconsistency on the meltano tap-mysql page stating that the license is Apache 2.0 however in Github it says that it is GPL 3.0. Which is the correct one?

👀 oh, where are you seeing that? The tap is MIT AFAICT: https://github.com/MeltanoLabs/tap-mysql/blob/3e05853f8258fb332c1ff487ad30ca421348a838/pyproject.toml#L22

@Edgar Ramírez (Arch.dev) sorry I didn't state - I was referring to the pipelinewise variant: https://github.com/transferwise/pipelinewise-tap-mysql https://hub.meltano.com/extractors/tap-mysql/

The Wise folks moved their Singer connectors to https://github.com/transferwise/pipelinewise, which is licensed Apache-2.0, but the license in the subdirectory is indeed AGPL-3: https://github.com/transferwise/pipelinewise/blob/3d8e7bc6214a6876ec3871ca4b4aca6bbe27ba17/singer-connectors/tap-mysql/LICENSE. I think the way we parse the license type from the repo doesn't take into account whether the connector is actually in a subdirectory.

@Edgar Ramírez (Arch.dev) Hi Edgar, where are the audit fields created in the sdk repository? Can this logic be replicated within the tap extractor?

That target is normally the one that adds the

_sdc

fields. Maybe using stream maps you could simulate some of the fields. The Wise tap only adds

_sdc_deleted_at

for log-based replication.

We are using meltanolabs variant - and I believe the stream maps doesn't work with batch processing?