Hi, Someone already use replication with

tap-redshift

(https://github.com/datadotworld/tap-redshift) ? We defined replication properties in catalog with

replication-method

and

replication-key

but elt method seems not consider the state. At each starting, the elt method return

INFO Building State from raw state {}

while at the end the tap seems create a bookmark like

2022-06-14T07:26:00.952438Z [info     ] Incremental state has been updated at 2022-06-14 07:26:00.952275.

2022-06-14T07:26:00.952663Z [debug    ] Incremental state: {'bookmarks': {'MA TABLE': {'replication_key': '_sdc_batched_at', 'version': 1655191553524, 'replication_key_value': '2022-06-13'}}, 'currently_syncing': None}

Do people use it? https://hub.meltano.com/extractors/tap-redshift Seems to point to yes, although not a lot of runs with meltano it seems?

Hi Derek,

- name: tap-redshift
  namespace: tap_redshift
  pip_url: git+<https://github.com/signal-ai/tap-redshift.git>
  executable: tap-redshift
  capabilities:
  - catalog
  - discover
  settings:
  - name: host
  - name: port
    value: my_port
  - name: dbname
    value: dwh
  - name: user
  - name: password
  - name: start_date
    value: '2022-06-10T00:00:00Z'
  - name: schema
    value: my_schema
  catalog: extract/redshift_catalog/redshift_catalog.json

Nice, so after you run

meltano elt tap-redshift target-csv --job_id=mon-id

could you then run

meltano state get mon-id

and show the contents?

We get the following result

2022-06-14T14:50:13.765961Z [info     ] Environment 'prod' is active

2022-06-14T14:50:14.135180Z [info     ] Found state from 2022-06-14 06:51:16.431545.

{"singer_state": {"bookmarks": {"<http://db.my|db.my>_<http://schema.my|schema.my>_table": {"replication_key": "_sdc_batched_at", "version": 1655189477777, "replication_key_value": "2022-06-13"}}, "currently_syncing": null}}

Ok so

<https://docs.meltano.com/reference/command-line-interface#examples>

meltano elt tap-redshift target-csv --state-id=mon-id > state.json

Would return the same

Oh thanks!!!! 😄 tap well work now. I didn't know we have to add capabilities at each tap because sometime tap replication work without specify capabilities.