cc: @TomasB We're having a problem where we are getting duplicate records (looks like probably the last record from the previous run). We're using postgres transferwise tap and target-bigquery adswerve target. We have replication mode set to INCREMENTAL. On the adswere Readme, they mention:

Upserting new rows into the table, using the primary key given by the tap connector (if it finds an old row with same key, updates it. Otherwise it inserts the new row)

We have set the:

table-key-properties:
              - id

extended

- name: prod
  config:
    plugins:
      extractors:
      - name: tap-postgres--inventory
        config:
          default_replication_method: INCREMENTAL
          metadata:
            public-*:
              replication-key: updated_date
              table-key-properties:
              - id

I thought this would solve the problem. Is there another field for the primary key I should be using instead of table-key-properties?

https://github.com/MeltanoLabs/Singer-Working-Group/issues/13 is a great write up of why this happens 😄

yep, you're right, that's the singer spec that takes a "at least once" approach, but yeah, the target is supposed to use the id for the upsert (or merge in GBQ case), which was why I was wondering if there is some different metadata tag or some other way to specify the primary key.