Hey all, I have a quick question if you could help. I've been looking for a way to handle hard deletes in my data source and the two solutions I've seen are using

ACTIVATE_VERSION

as my singer table replication method, or using the Target Postgres variant transferwise to take advantage of the

_sdc_

metadata columns, then use dbt snapshots and so on. My question is, has anyone found/know of a work around to completely refresh a tap's stream table every sync? I could then just snapshot this table and capture deletes that way. Also are there any other methods for capturing hard deletes that I've missed? Wanted to see if this was a think before I commit to the whole

ACTIVATE_VERSION

refactor that I think I'm going to need to do. Thanks

If you used FULL_TABLE from your tap on every stream, and had your target table get populated with all the data from your source. And you could guarantee that if your tap succeeds it has all the data it needs (and you could make sure the target doesn't write your data without the tap finishing) You could then upsert from that table into your stage table. The accepted method is to use the meta data columns. Splunking gitlab's dbt project is fun for this kinda stuff too https://gitlab.com/gitlab-data/analytics/-/blob/master/transform/snowflake-dbt/models/sources/zendesk/zendesk_groups_source.sql Zendesk source is from stitch, they use the is_deleted flag. Target is probably target-snowflake

Thanks for the info :)