#3556 fix: Ensure parent stream records are emitted in auto-generated tap tests
Pull request opened by
edgarrmondragon
sdk/singer_sdk/tap_base.py
Lines 284 to 286 in</meltano/sdk/commit/4a5dd894d69cc068d140acea6e15fa4040831611|4a5dd89>
| if not stream.child_streams: # pragma: no branch |
| ------------------------------------------------------------------- |
| # Initialize streams' record limits before beginning the sync test. |
| stream.ABORT_AT_RECORD_COUNT = dry_run_record_limit |
Related
•
#3029
•
#3030
•
#3037
•
#3545
Summary by Sourcery
Adjust tap dry-run behavior to cap record counts on all streams while ensuring parent streams continue emitting records when child streams hit their record limits.
Bug Fixes:
• Prevent auto-generated dry-run syncs from aborting parent streams when child streams reach their record limit by catching child abort exceptions and stopping only remaining child syncs.
• Apply the dry-run record limit consistently to both parent and child streams so all streams are subject to the same cap.
Summary by Sourcery
Ensure dry-run syncs apply record limits consistently across parent and child streams without preventing parent records from being emitted when children hit their cap.
Bug Fixes:
• Apply the dry-run record limit to all streams, including parents and children, instead of only non-child streams.
• Prevent child stream aborts due to dry-run record limits from stopping sibling or parent stream processing by catching abort exceptions during child sync.
Tests:
• Add a regression test verifying that parent and sibling records are still emitted, and child records are correctly capped, when a child stream hits the dry-run record limit.
meltano/sdk