GitHub
06/06/2025, 5:53 PMACTIVATE_VERSION messages for streams with FULL_TABLE replication
Pull request opened by edgarrmondragon
• Integration test plan: E2E snaphots accomplish this
• Spec compliance tests. Do snapshots cover this?
• Docs: #3079
### Related
• Closes #18
• https://github.com/meltano/sdk/discussions/3071
• singer-io/getting-started#71
• meltano/meltano#2463 (https://hub.meltano.com/singer/docs/#activate-version)
---
📚 Documentation preview 📚: https://meltano-sdk--2686.org.readthedocs.build/en/2686/
## Summary by Sourcery
Emit ACTIVATE_VERSION messages and attach version metadata for FULL_TABLE replication streams; adjust replication key behavior, batch messages, and sample mapper logic; and bolster snapshot-based tests for tap→mapper→target workflows.
New Features:
• Enable taps to emit ACTIVATE_VERSION messages and include version metadata for streams using FULL_TABLE replication
Enhancements:
• Use tap initialization time for replication key signposts in timestamp-based replication
• Introduce a version field in RecordMessage and SDKBatchMessage to track table versions
• Skip RemoveRecordTransform when mapping ActivateVersionMessage in sample mapper
• Update codespell config to ignore JSONL snapshot files
Tests:
• Add snapshot-driven end-to-end tests for tap→mapper→target flows with time-machine for deterministic timestamps
• Expand and update sample and core test suites with new fixtures and snapshot files
## Summary by Sourcery
Enable taps to optionally emit ACTIVATE_VERSION messages and include version metadata for FULL_TABLE replication streams, extend core stream and message classes to support version tracking, update sample mapper and capabilities, and add snapshot-driven tests for tap→mapper→target workflows.
New Features:
• Add opt-in support for emitting ACTIVATE_VERSION messages with version metadata for FULL_TABLE replication streams.
Enhancements:
• Add a version field to RecordMessage and SDKBatchMessage to track table version.
• Use tap initialization timestamp as the replication key signpost for timestamp-based replication.
• Introduce emit_activate_version_messages config property and update tap capabilities to include ACTIVATE_VERSION.
• Skip RemoveRecordTransform when mapping ActivateVersionMessage in the sample mapper.
• Update codespell configuration to ignore JSONL snapshot files.
Tests:
• Add snapshot-driven end-to-end tests for CSV and SQLite targets covering activate_version, hard delete, and missing properties scenarios.
• Refactor tests to use pytest snapshots, caplog, time_machine fixture, and deterministic table names using request.node.name.
Chores:
• Adjust existing test setup to use pytest snapshot plugin and remove UUID-based table naming.
meltano/sdkGitHub
06/06/2025, 5:53 PMACTIVATE_VERSION messages for streams with FULL_TABLE replication
Pull request ready for review by edgarrmondragon
meltano/sdkGitHub
06/06/2025, 5:54 PM