GitHub
09/02/2026, 12:44 AMself.paginator) across all method scopes within the REST stream class, not just within request_records. This eliminates the need to duplicate pagination configuration values across different methods.
Key Changes
• Added pagination instance attribute - _current_paginator with proper lifecycle management
• Implemented context manager pattern - Clean creation and cleanup of paginator per request cycle
• Enhanced method documentation - Updated get_url_params with usage examples
• Comprehensive test coverage - 4 new test cases covering all aspects of the feature
Benefits
• Reduces code duplication - Access pagination config from any method during request processing
• Maintains backward compatibility - All existing implementations continue to work unchanged
• Fresh paginator per context - Each request_records call gets a new paginator instance
• Improved developer experience - Clear error messages when accessing paginator incorrectly
Usage Example
Before (with duplication):
class MyStream(RESTStream):
PAGE_SIZE = 25
def get_new_paginator(self):
return BaseOffsetPaginator(start_value=0, page_size=self.PAGE_SIZE)
def get_url_params(self, context, next_page_token):
return {"limit": self.PAGE_SIZE} # Duplicated constant
After (no duplication):
class MyStream(RESTStream):
def get_new_paginator(self):
return BaseOffsetPaginator(start_value=0, page_size=25)
def get_url_params(self, context, next_page_token):
# Access paginator configuration directly
try:
limit = self.paginator.page_size # If accessible
except AttributeError:
limit = 25 # fallback
return {"limit": limit}
Testing
• All existing tests continue to pass
• New comprehensive test suite validates:
• Paginator accessibility across multiple methods
• Fresh paginator creation per request
• Proper lifecycle cleanup
• Backward compatibility
• Error handling when accessed incorrectly
🤖 Generated with Claude Code
Summary by Sourcery
Make the pagination object available across all REST stream methods by introducing a context-managed paginator property while preserving existing behaviors
New Features:
• Expose self.paginator property during request_records lifecycle
• Introduce _paginator_context context manager to handle paginator creation and cleanup
Enhancements:
• Refactor request_records to use the paginator context manager
• Update get_url_params documentation to reference the new paginator property
Documentation:
• Enhance RESTStream docstrings to describe self.paginator availability and usage
Tests:
• Add tests for paginator accessibility in multiple methods
• Add tests for fresh paginator creation per request_records call
• Add tests for paginator lifecycle cleanup
• Add tests for fallback when get_new_paginator returns None
• Add tests for backward compatibility with legacy pagination patterns
meltano/sdkGitHub
09/02/2026, 12:44 AM