Hi, I'm using this loader: https://hub.meltano.com/loaders/target-redshift--ticketswap/. I have error message:

2024-08-15T18:11:07.139236Z [info     ]     sink.activate_version(message_dict["version"]) cmd_type=elb consumer=True job_name=dev:tap-mysql-to-target-redshift name=target-redshift producer=False run_id=ba769159-d62b-4e50-b99a-d6b44a0d4b06 stdio=stderr string_id=target-redshift
2024-08-15T18:11:07.139510Z [info     ]   File "/home/ec2-user/dev_qoala_policy_service_development/.meltano/loaders/target-redshift/venv/lib/python3.9/site-packages/singer_sdk/sinks/sql.py", line 382, in activate_version cmd_type=elb consumer=True job_name=dev:tap-mysql-to-target-redshift name=target-redshift producer=False run_id=ba769159-d62b-4e50-b99a-d6b44a0d4b06 stdio=stderr string_id=target-redshift
2024-08-15T18:11:07.139870Z [info     ]     self.connector.prepare_column( cmd_type=elb consumer=True job_name=dev:tap-mysql-to-target-redshift name=target-redshift producer=False run_id=ba769159-d62b-4e50-b99a-d6b44a0d4b06 stdio=stderr string_id=target-redshift
2024-08-15T18:11:07.140125Z [info     ] TypeError: prepare_column() missing 1 required positional argument: 'cursor' cmd_type=elb consumer=True job_name=dev:tap-mysql-to-target-redshift name=target-redshift producer=False run_id=ba769159-d62b-4e50-b99a-d6b44a0d4b06 stdio=stderr string_id=target-redshift

source is

tap-mysql

, target is

target-redshift

. this is another error:

2024-08-15T18:16:32.974038Z [info     ] TypeError: prepare_column() missing 1 required positional argument: 'cursor' cmd_type=elb consumer=True job_name=dev:tap-mysql-to-target-redshift name=target-redshift producer=False run_id=50ff1ec0-420f-4a95-b61b-232224613201 stdio=stderr string_id=target-redshift
2024-08-15T18:16:33.012738Z [info     ] time=2024-08-15 18:16:33 name=singer level=INFO message=METRIC: {"type": "counter", "metric": "record_count", "value": 6946, "tags": {"database": "some_db", "table": "some_table"}} cmd_type=elb consumer=False job_name=dev:tap-mysql-to-target-redshift name=tap-mysql producer=True run_id=50ff1ec0-420f-4a95-b61b-232224613201 stdio=stderr string_id=tap-mysql
2024-08-15T18:16:33.013479Z [info     ] time=2024-08-15 18:16:33 name=singer level=INFO message=METRIC: {"type": "timer", "metric": "job_duration", "value": 3.8889615535736084, "tags": {"job_type": "sync_table", "database": "some_db", "table": "some-table", "status": "succeeded"}} cmd_type=elb consumer=False job_name=dev:tap-mysql-to-target-redshift name=tap-mysql producer=True run_id=50ff1ec0-420f-4a95-b61b-232224613201 stdio=stderr string_id=tap-mysql

meltano.yml

:

version: 1
default_environment: dev
project_id: f993d6ab-94d6-4504-9857-20cbe86700bd
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-mysql
    variant: transferwise
    pip_url: pipelinewise-tap-mysql
    config:
      database: some_db
      engine: mysql
      host: localhost
      port: 3307
      user: root
    select:
    - some_schema-some_table.*
    metadata:
      some_schema-some_table:
        replication-method: FULL_TABLE
  loaders:
  - name: target-redshift
    variant: ticketswap
    pip_url: git+<https://github.com/TicketSwap/target-redshift.git>
    config:
      aws_redshift_copy_role_arn: arn:aws:iam::xxxxxxxxxx:role/AWSServiceRoleRedshift
      batch_size_rows: 100000
      dbname: dev
      default_target_schema: meltano_schema
      enable_iam_authentication: false
      host: <http://cluster-identifier.xxxxxxx.ap-southeast-1.redshift.amazonaws.com|cluster-identifier.xxxxxxx.ap-southeast-1.redshift.amazonaws.com>
      port: '5439'
      s3_bucket: some-s3-bucket-name
      user: adminredshift
elt:
  buffer_size: 52428800

2024-08-15T18:11:07.139236Z [info     ]     sink.activate_version(message_dict["version"]) cmd_type=elb consumer=True job_name=dev:tap-mysql-to-target-redshift name=target-redshift producer=False run_id=ba769159-d62b-4e50-b99a-d6b44a0d4b06 stdio=stderr string_id=target-redshift
2024-08-15T18:11:07.139510Z [info     ]   File "/home/ec2-user/dev_qoala_policy_service_development/.meltano/loaders/target-redshift/venv/lib/python3.9/site-packages/singer_sdk/sinks/sql.py", line 382, in activate_version cmd_type=elb consumer=True job_name=dev:tap-mysql-to-target-redshift name=target-redshift producer=False run_id=ba769159-d62b-4e50-b99a-d6b44a0d4b06 stdio=stderr string_id=target-redshift
2024-08-15T18:11:07.139870Z [info     ]     self.connector.prepare_column( cmd_type=elb consumer=True job_name=dev:tap-mysql-to-target-redshift name=target-redshift producer=False run_id=ba769159-d62b-4e50-b99a-d6b44a0d4b06 stdio=stderr string_id=target-redshift
2024-08-15T18:11:07.140125Z [info     ] TypeError: prepare_column() missing 1 required positional argument: 'cursor' cmd_type=elb consumer=True job_name=dev:tap-mysql-to-target-redshift name=target-redshift producer=False run_id=ba769159-d62b-4e50-b99a-d6b44a0d4b06 stdio=stderr string_id=target-redshift

That's a target bug, why are you using that variant?

I'm using that variant because using meltano SDK and still active. Ok I'll try this variant https://hub.meltano.com/loaders/target-redshift--transferwise/. Thank you.

Can you submit an issue to that repo as well? Ideally a PR to fix it but an issue is a good start!

I think this is triggered because tap-mysql implements

ACTIVATE_VERSION

and so the framework goes through this code path: https://github.com/meltano/sdk/blob/6b5a44aa7d1d0b628ddc039698230b4b790a92b6/singer_sdk/sinks/sql.py#L379-L387

https://github.com/TicketSwap/target-redshift/blob/main/target_redshift/connector.py#L274-L280 they overwrote the definition of prepare_column so the target should also override activate version imo There's of course deeper questions about library design and which functions folks should override.

We also overrode those functions for a different reason in target-postgres

Which make me think the default implementation is not great 😬

Issue submitted. now I'm using this https://hub.meltano.com/loaders/target-redshift--transferwise/ and did some changes because of psycopg2-binary version and working properly. Thank you.