I have another question, not as blocking but more trying to understand how this works, i've tried looking for docs on singer spec or meltano but didn't find much,; I see there are a couple of metadata columns added with targets such as target-snowflake from datamill, but this column stays empty with my tap:

_SDC_TABLE_VERSION

(I see the logic to populate it comes from target-postgres) Is there a way to specify the table version somewhere in the singer sdk for taps, or is this something to specify in the target directly? Is this even worth pursuing or just some compatibility thing?

This is connected with an unofficial extension known as “activate version”. For targets which support it, the “activate version” message can be used to trigger deprecation of previously-received records - records prior to a specific version number. I don’t have specific documentation reference for that feature, but we may add this when we get to documenting common singer spec extensions.

That's great to know, i think i don't need this feature for my particular singer tap, it's mostly i wanted to understand why the target was creating this null column 😂 thanks a lot @aaronsteers!!

On our radar here: Add documentation on _sdc_ metadata columns. (#2765) · Issues · meltano / Meltano · GitLab And I’ve linked to this conversation so we make sure to cover this when addressing other spec extensions and sdc columns.

You are on top of everything! Amazing

😄 Very kind. But yeah, we definitely want to demystify and improve accessibility of documentation where ever we can.

As general feedback the docs on Meltano and singer SDK are amazing, i was going crazy trying to use generic Singer stuff, as a suggestion more examples on common scenarios would be amazing, though already the examples that are available are super useful on understanding how people integrate APIs with it

Awesome - thanks @pablo_seibelt for the feedback. The more people build on it, the more examples we should be able to gather. We are also thinking of adding an indicator to the #C01UGBSJNG5 which would let you identify SDK-based taps more quickly from those which are published there.

Thanks!