#3058 feat(taps): Queue parent contexts and sync child streams only when the queue is full
Pull request opened by
edgarrmondragon
## Summary by Sourcery
Introduce batching of child stream synchronization by queueing parent contexts and flushing them when the queue reaches a configurable maximum or at the end of a parent sync.
New Features:
• Enable delayed syncing of child streams by queuing parent contexts instead of immediate synchronization.
Enhancements:
• Add a configurable QUEUE_MAX_SIZE attribute (default 1000) to control batch size of queued contexts.
• Implement an internal _child_context_queue and a _flush_child_context_queue method to process queued contexts in bulk.
• Modify _sync_children and _sync_records to enqueue contexts and trigger flushes when the queue is full or after parent sync ends.
Tests:
• Adjust existing parent-child tests to override QUEUE_MAX_SIZE and verify batched flushing behavior.
meltano/sdk