Hey folks, I'm trying to run

meltano run tap-csv target-mysql

. But I get a

MySQLdb.OperationalError: (1071, 'Specified key was too long; max key length is 3072 bytes')

. Happens directly due to:

2024-08-23T12:17:10.246867Z [info     ] [SQL:                          cmd_type=elb consumer=True job_name=dev:tap-csv-to-target-mysql name=target-mysql producer=False run_id=e0b3b46e-626d-4d3a-83e6-abce06fe81f5 stdio=stderr string_id=target-mysql
2024-08-23T12:17:10.247048Z [info     ] CREATE TABLE test (            cmd_type=elb consumer=True job_name=dev:tap-csv-to-target-mysql name=target-mysql producer=False run_id=e0b3b46e-626d-4d3a-83e6-abce06fe81f5 stdio=stderr string_id=target-mysql
2024-08-23T12:17:10.247315Z [info     ] 	id VARCHAR(1000) NOT NULL,    cmd_type=elb consumer=True job_name=dev:tap-csv-to-target-mysql name=target-mysql producer=False run_id=e0b3b46e-626d-4d3a-83e6-abce06fe81f5 stdio=stderr string_id=target-mysql
2024-08-23T12:17:10.247531Z [info     ] 	text VARCHAR(1000),           cmd_type=elb consumer=True job_name=dev:tap-csv-to-target-mysql name=target-mysql producer=False run_id=e0b3b46e-626d-4d3a-83e6-abce06fe81f5 stdio=stderr string_id=target-mysql
2024-08-23T12:17:10.247830Z [info     ] 	value VARCHAR(1000),          cmd_type=elb consumer=True job_name=dev:tap-csv-to-target-mysql name=target-mysql producer=False run_id=e0b3b46e-626d-4d3a-83e6-abce06fe81f5 stdio=stderr string_id=target-mysql
2024-08-23T12:17:10.247994Z [info     ] 	sdc_source_file_ VARCHAR(1000), cmd_type=elb consumer=True job_name=dev:tap-csv-to-target-mysql name=target-mysql producer=False run_id=e0b3b46e-626d-4d3a-83e6-abce06fe81f5 stdio=stderr string_id=target-mysql
2024-08-23T12:17:10.248236Z [info     ] 	sdc_source_file_mtime_ DATETIME, cmd_type=elb consumer=True job_name=dev:tap-csv-to-target-mysql name=target-mysql producer=False run_id=e0b3b46e-626d-4d3a-83e6-abce06fe81f5 stdio=stderr string_id=target-mysql
2024-08-23T12:17:10.249244Z [info     ] 	sdc_source_lineno_ BIGINT,    cmd_type=elb consumer=True job_name=dev:tap-csv-to-target-mysql name=target-mysql producer=False run_id=e0b3b46e-626d-4d3a-83e6-abce06fe81f5 stdio=stderr string_id=target-mysql
2024-08-23T12:17:10.249495Z [info     ] 	CONSTRAINT `test_PK` PRIMARY KEY (id) cmd_type=elb consumer=True job_name=dev:tap-csv-to-target-mysql name=target-mysql producer=False run_id=e0b3b46e-626d-4d3a-83e6-abce06fe81f5 stdio=stderr string_id=target-mysql

I have attached the meltano.yml and the error logs as well. Is there any way to change that VARCHAR(1000) to something less, like 255; Any suggestions?

The target seems to default to 1000 when

maxLength

is not present: https://github.com/thkwag/target-mysql/blob/c84fe01850b0470d39e4d0c56fd884e2019174c4/target_mysql/sinks.py#L195-L196 tap-postgres doesn't emit that property, but we are going to fix that upstream in a future version of the SDK (https://github.com/meltano/sdk/pull/2618). In the meantime, can you try overriding the

schema

for the offending field(s):

extractors:
- name: tap-postgres
  schema:
    some_stream_id:
      created_at:
        maxLength: date-time

https://docs.meltano.com/concepts/plugins/#schema-extra

Oh alright, thanks