<#3050 feat(taps): Allow SQL stream implementation...
# meltano-repo-activity
g
#3050 feat(taps): Allow SQL stream implementations to override the
SELECT
statement filter and limit clauses without modifying extraction logic
Pull request opened by edgarrmondragon ## Related • MeltanoLabs/tap-postgres#628 ## Summary by Sourcery Provide extension points for customizing SQL query generation by extracting replication filtering and limit logic into separate hook methods (
build_query
,
apply_replication_filter
, and
apply_abort_query_limit
) and refactoring
get_records
to use them. New Features: • Add
build_query
method to allow streams to override base SELECT query • Introduce
apply_replication_filter
hook to encapsulate replication key ordering and filtering • Introduce
apply_abort_query_limit
hook to encapsulate record limit enforcement Enhancements: • Refactor
get_records
to delegate query construction to new hook methods • Mark unsupported partitioning branch with
# pragma: no cover
## Summary by Sourcery Extract SQL filter and limit logic into overrideable hooks and update record retrieval to use them New Features: • Add apply_query_filters hook for customizing replication key-based WHERE and ORDER BY clauses • Add apply_query_limit hook for customizing ABORT_AT_RECORD_COUNT-based LIMIT clause Enhancements: • Refactor get_records to delegate filtering and limiting to the new hooks • Annotate unsupported partitioning branch with a coverage pragma meltano/sdk