<#3649 fix(targets): Reset duplicate-merged tally ...
# meltano-repo-activity
g
#3649 fix(targets): Reset duplicate-merged tally between batches Pull request opened by edgarrmondragon Summary •
Sink.mark_drained()
reset
_batch_records_read
to 0 but never reset
_batch_dupe_records_merged
• On each subsequent drain,
tally_record_written()
subtracted all accumulated duplicates (not just those from the current batch), making
_total_records_written
increasingly wrong • Fixed by resetting
_batch_dupe_records_merged = 0
alongside
_batch_records_read = 0
in
mark_drained()
• Added parametrized regression test that verifies all records are written across multiple drain cycles when
batch_size_rows < total_records
Test plan •
uv run pytest tests/core/test_target_base.py -v
Summary by Sourcery Reset duplicate-merge batch counters correctly between drain cycles and add regression coverage to ensure all records are written when using row-based batching. Bug Fixes: • Reset the per-batch duplicate-merged counter when draining so total records written is tallied correctly across multiple batches. Tests: • Add a parametrized regression test to verify no records are lost and batch counts are correct when batch_size_rows is smaller than the total number of records. meltano/sdk