I don’t have the time (or the knowledge) to work o...
# singer-tap-development
h
I don’t have the time (or the knowledge) to work on this right now, but I want to mention it in case anyone else might find it interesting: Deleting data usually requires log-based replication, but it should be possible to do the same using snowflake streams, because they are practically a log and marks rows as “insert” or “delete” in the correct order. I don’t know if it makes sense to add this as a feature to the existing tap or to create a tap-snowflake-streams, but it would be a fairly nice way of doing replication from snowflake.
e
cc @ken_payne
a
Agreed - but with a couple caveats. This would only apply for tables that are modified with a traditional row 'load' and 'delete' paradigm (basically oltp). However, dbt-managed tables are often drop-and-replace in each build, so those final-created tables from a dbt-managed process may not work as expected.
I don't know a ton about Snowflake streams but I would just verify the load/update/delete/create patterns for the desired source tables/streams.
h
I agree with that caveat, in a dbt context only snapshots and incremental models make sense. When tables are replaced, any streams based on them are dropped so the E/L would fail unless the stream is recreated.
In any case, this was more of a loose thought on my side, not a real feature request.