My snowflake data source deletes account-services rather than marking them as cancelled. Literally deletes them from their DB. If I did a full table replication on this could I configure it to not delete records that are no longer in the source but just mark them as deleted?

Sounds like you might need the 'activate_version' functionality? Not supported by all taps and targets though. https://github.com/meltano/sdk/pull/2686

How can I know if the snowflake tap has that?

I don't use that tap myself but perhaps @Edgar Ramírez (Arch.dev) could advise. What target are you using? Alternatively could you use an 'append only' strategy? and handle duplicates downstream? Any primary keys that weren't synced in the 'latest' sdc run you could identify as deleted. I've used that with success in the past.

Postgres is my target

The meltanolabs variant of target-postgres does support activate version (I use it in prod). I think for your tap it doesn't look supported by default, it only got released in a recent singer SDK version.

How hard would it be for me to do a PR to add that to tap-snowflake?

If you're interested in contributing @Tanner Wilcox, it should be as easy as: 1. bump the SDK version to 0.47 or 0.48 here: https://github.com/MeltanoLabs/tap-snowflake/blob/b415fe069728dc4a5d7047b67d911e87e56b1ab9/pyproject.toml#L20 2. run

poetry lock

3. configure the tap with

emit_activate_version_messages: true

4. confirm that it works as expected

@Edgar Ramírez (Arch.dev) I think that enabling this turns it on for all streams that you are syncing via 'FULL_TABLE', is that right? It can't be configured on a stream by stream basis even if you know the streams where hard-deletes can occur in the source

That's correct, need to create an issue for that. I'll do that in the coming days and tie it to the general "per-stream" configuration support.

Wait, so I can turn this on now for tap-snowflake?

No, the tap is in old version of the SDK that doesn't support it

@Edgar Ramírez (Arch.dev) re the 'general per-stream configuration support', is there any existing issues on this I could read up on?

There's https://github.com/meltano/sdk/issues/1350