During the incremental replication from PostgreSQL to BigQuery, if there are no new records, then one entry is getting copied in every time I run the meltano elt command. Not sure why it is copying one entry only. Is there way to avoid it by adding primary key constraint on the replicated table in the BigQuery? The id is uuid and created_at is timestamp. Below is the replication details

replication-method: INCREMENTAL
replication-key: created_at

Output is

It is expected that extractor (taps) repeat the last record of a previous run in a subsequent incremental run: https://www.stitchdata.com/docs/replication/replication-methods/key-based-incremental#limitation-3--replication-key-inclusivity Loaders (targets) are expected to deal with this by upserting based on the primary key, but I think target-bigquery currently always appends (https://gitlab.com/meltano/meltano/-/issues/2504). The workaround is to explicitly deduplicate using a post-load dbt transformation (https://gitlab.com/meltano/meltano/-/issues/2670).

@douwe_maan Thank you for your response. Let me go through these. I will come back if I have any questions.