I'm using the tap-mysql transferwise variant to ingest data from some MySQL views. No primary keys are define in these views, there are no reliable timestamps for incremental replication and I also don't have access to the binary logs for log-based incremental replication. I'm left with full table replication, that's fine. But since there are no primary key defined, when the data is loaded in the target (target-snowflake transferwise variant if that matter), each time the pipeline is triggered I get a new full copy of the views because it only use the

COPY INTO

statement in Snowflake. Is there a way to pass a composite primary key to the tap so that the operation in Snowflake is a

MERGE

on the composite key instead of a full insert? I search to figure if there was a way to add a composite primary key in the tap and I found this thread. Based on this, it seems like I can configure the

table-key-properties

of the entities, but this haven't worked for me so far, it still

COPY INTO

instead of

MERGE

. Here is the config I have:

plugins:
  extractors:
  - name: tap-mysql
    variant: transferwise
    pip_url: pipelinewise-tap-mysql
    metadata:
      my_entity:
        table-key-properties:
        - Country
        - DeviceType
        - EndDate
        - Organization
        - SerialNumber
        - StartDate

Can you try with

view-key-properties

instead?

Thank you, it worked as expected, full table being

merge

on the composite key defined in the tap

view-key-properties

.