I need help understanding how to do incremental re...
# troubleshooting
t
I need help understanding how to do incremental replication while also overwriting the previous record
1
Here's my meltano.yml for a snowflake tap:
Copy code
plugins:
  extractors:
  - name: sonar
    inherit_from: tap-snowflake
    variant: meltanolabs
    config:
      account: 1233
      user: ext_3c255d90f6
      password: ${SONAR_PASS}
      warehouse: ETL_WAREHOUSE
      database: TENANT_DATA_SNOWPIPE
      schema: PUBLIC
    metadata:
      public-addresses_ext_3c255d82059647efba26c0996cec90f6:
        replication-method: INCREMENTAL
        replication-key: updated_at
        updated_at:
          is-replication-key: true
I want it to only pull a record from that address table if the updated_at is new but I want it to replace the previous record based on the ID
I tried adding
table-key-properties: [id]
but that didn't seem to do anything
h
try this?
Copy code
metadata:
      public-addresses_ext_3c255d82059647efba26c0996cec90f6:
        key_properties: ['id']
        replication-key: updated_at
        replication-method: INCREMENTAL
also if this is exactly what you had
table-key-properties: [id]
, quotes needed?
t
Getting rid of the 'table-' works