#3545 feat: Attempt to sync all streams instead of crashing on the first error
Pull request opened by
edgarrmondragon
Stub
Summary by Sourcery
Track per-stream sync results and allow tap runs to continue syncing remaining streams after individual failures while reporting an aggregate outcome.
New Features:
• Introduce a SyncResult enum to represent and combine per-stream sync outcomes and map them to process exit codes.
• Return an aggregate SyncResult from Tap.sync_all and use it to set the tap process exit code instead of always exiting successfully.
Bug Fixes:
• Ensure errors in child streams mark the parent stream as partially successful instead of aborting remaining children and records.
• Convert unexpected exceptions during stream sync into lifecycle abort exceptions so they are consistently handled and logged.
Enhancements:
• Log a one-line sync outcome per stream and improve error log messages during sync to include exception details.
• Extend parent/child stream typing with Context and Record aliases and apply override annotations for better type safety.
• Adjust typing-extensions dependency marker for Python 3.13 compatibility.
Tests:
• Add unit tests and snapshot tests covering SyncResult behavior and tap behavior when some streams fail while others continue syncing, including incremental and parent/child scenarios.
meltano/sdk