<#1663 feat: Add built-in paginator classes for da...
# meltano-repo-activity
g
#1663 feat: Add built-in paginator classes for date and datetime ranges Issue created by edgarrmondragon Feature scope Taps (catalog, state, stream maps, etc.) Description It's relatively common for APIs to support requesting data with a start and end date, but sometimes the API limits the size of the date range (e.g. to 1 week maximum) so it's necessary to iterate over the date range: • 2023-05-01 - 2023-05-07 • 2023-05-08 - 2023-05-15 • ... A new implementation of
BaseAPIPaginator
could help with this. Examples I'll gather examples I encounter in the wild, so we can implement a somewhat generic base class. Weather API The
/history.json
endpoint supports
dt
and
end_dt
parameter: 'end_dt' should be on or after 1st Jan, 2010 in yyyy-MM-dd format (i.e. dt=2010-01-01). 'end_dt' should be greater than 'dt' parameter and difference should not be more than 30 days between the two dates. --- Related: • #2318 meltano/sdk