<#3245 feat: Make pagination object available in m...
# meltano-repo-activity
g
#3245 feat: Make pagination object available in more scopes of REST stream class Pull request opened by edgarrmondragon Summary Resolves #1606 This PR makes the pagination object available as an instance attribute (
self.paginator
) across all method scopes within the REST stream class, not just within
request_records
. This eliminates the need to duplicate pagination configuration values across different methods. Key Changes • Added pagination instance attribute -
_current_paginator
with proper lifecycle management • Implemented context manager pattern - Clean creation and cleanup of paginator per request cycle • Enhanced method documentation - Updated
get_url_params
with usage examples • Comprehensive test coverage - 4 new test cases covering all aspects of the feature Benefits • Reduces code duplication - Access pagination config from any method during request processing • Maintains backward compatibility - All existing implementations continue to work unchanged • Fresh paginator per context - Each
request_records
call gets a new paginator instance • Improved developer experience - Clear error messages when accessing paginator incorrectly Usage Example Before (with duplication): class MyStream(RESTStream): PAGE_SIZE = 25 def get_new_paginator(self): return BaseOffsetPaginator(start_value=0, page_size=self.PAGE_SIZE) def get_url_params(self, context, next_page_token): return {"limit": self.PAGE_SIZE} # Duplicated constant After (no duplication): class MyStream(RESTStream): def get_new_paginator(self): return BaseOffsetPaginator(start_value=0, page_size=25) def get_url_params(self, context, next_page_token): # Access paginator configuration directly try: limit = self.paginator.page_size # If accessible except AttributeError: limit = 25 # fallback return {"limit": limit} Testing • All existing tests continue to pass • New comprehensive test suite validates: • Paginator accessibility across multiple methods • Fresh paginator creation per request • Proper lifecycle cleanup • Backward compatibility • Error handling when accessed incorrectly 🤖 Generated with Claude Code Summary by Sourcery Make the pagination object available across all REST stream methods by introducing a context-managed paginator property while preserving existing behaviors New Features: • Expose self.paginator property during request_records lifecycle • Introduce _paginator_context context manager to handle paginator creation and cleanup Enhancements: • Refactor request_records to use the paginator context manager • Update get_url_params documentation to reference the new paginator property Documentation: • Enhance RESTStream docstrings to describe self.paginator availability and usage Tests: • Add tests for paginator accessibility in multiple methods • Add tests for fresh paginator creation per request_records call • Add tests for paginator lifecycle cleanup • Add tests for fallback when get_new_paginator returns None • Add tests for backward compatibility with legacy pagination patterns meltano/sdk