GitHub
05/27/2026, 6:45 PMSink.mark_drained() reset _batch_records_read to 0 but never reset _batch_dupe_records_merged
• On each subsequent drain, tally_record_written() subtracted all accumulated duplicates (not just those from the current batch), making _total_records_written increasingly wrong
• Fixed by resetting _batch_dupe_records_merged = 0 alongside _batch_records_read = 0 in mark_drained()
• Added parametrized regression test that verifies all records are written across multiple drain cycles when batch_size_rows < total_records
Test plan
• uv run pytest tests/core/test_target_base.py -v
Summary by Sourcery
Reset duplicate-merge batch counters correctly between drain cycles and add regression coverage to ensure all records are written when using row-based batching.
Bug Fixes:
• Reset the per-batch duplicate-merged counter when draining so total records written is tallied correctly across multiple batches.
Tests:
• Add a parametrized regression test to verify no records are lost and batch counts are correct when batch_size_rows is smaller than the total number of records.
meltano/sdkGitHub
05/27/2026, 7:58 PM