<#3400 WIP: Redesign sink architecture> Pull reque...
# meltano-repo-activity
g
#3400 WIP: Redesign sink architecture Pull request opened by edgarrmondragon on <!date^1764738898^{date_short}|2025-12-03T05:14:58Z> Summary by Sourcery Introduce a modern, type-safe sink loading architecture while preserving backwards compatibility with existing dict-based sink implementations. New Features: • Add load_batch and load_record entry points as the recommended modern APIs for batch and record sinks, alongside legacy process_* methods. • Introduce BatchContext as a strongly-typed batch context object, including helpers to convert to and from legacy dict contexts. • Provide pluggable batch accumulation strategies (count, time window, size, hybrid, and no-batch) via a new BatchStrategy abstraction. • Expose new sink protocols (IRecordLoader, IBatchLoader, IPreprocessable, IVersionable, IBatchable) for clearer, typed interfaces and extension points. • Extend the CSV target sink to demonstrate the modern load_batch pattern using BatchContext. Enhancements: • Delay SQL sink table preparation to the first batch rather than during setup to avoid data loss when multiple schema messages arrive for the same stream. • Export the new BatchContext, batch strategy classes, and sink protocol interfaces from the sinks package for easier consumption. • Improve sink base class docs and behavior so that subclasses must override either the modern load_* methods or the legacy process_* methods, with clearer error messages and guidance. Tests: • Add tests covering modern vs legacy load methods, BatchContext conversion, and error behavior when sinks fail to override any loading method. meltano/sdk