#3843 refactor: Decode Arrow files lazily
Pull request opened by
edgarrmondragon
SSIA
Summary by Sourcery
Process Arrow batch files lazily in bounded chunks to avoid loading entire files into memory.
Enhancements:
• Decode Arrow batch files lazily and process records incrementally in bounded chunks to reduce memory usage for large files.
Tests:
• Add coverage for lazy Arrow decoding and chunked batch processing.
meltano/sdk