Hello team! I’m testing meltano here and basically I have a tap-mongodb and target-snowflake my two biggest issues right now are: • when I delete a document in mongodb, i don’t see any flag (

_sdc_deleted_at

) indicating so nor the row is deleted from snowflake (which i kind of expected) - any thoughts on how to fix that? • when I sink the data to snowflake, my table has 4 columns

_id 
_sdc_batched_at
_sdc_deleted_at
_sdc_extracted_at
document

which I also expected because I tested with target-jsonl - is there any way I could flatten the field

document

in order to my table to have the collection’s fields directly in snowflake? bravo to the team, I really like the tool, trying to get familiarized with it since dbt coalesce 🙂

Hi @marcel_cheab - I'm not able to 100% remember how the deleted_at is populated. From memory this is a feature of pipeline wise targets, and it could only work with log based / full table I guess. So might be helpful to share your tap_mongo configuration. For the 2nd challenge. Flattening ‘document’ is a job for a mapper (which operates between the tap and target) https://sdk.meltano.com/en/latest/stream_maps.html

hey @aaron_phethean, I’ll take a look at the documentation you sent, thank you for now 🙂

Hi @marcel_cheab Did you ever figure out how to handle the deleted MongoDB documents? I have a similar need to understand in the target what was deleted. Thank you!