Any suggestion on how to handle an API which has date based pagination using

since

and

until

, so those exist as params to my query, but they are not present in the API response. Is there any way to reliably access the params sent in a post-query method like

post_process

or

get_records

? Or would I need to persist those values to instance variables (self.since, self.until etc) on each iteration?

Maybe something like this (thread unrelated): https://meltano.slack.com/archives/C068YBV6KEK/p1744302272258219?thread_ts=1744255144.140489&cid=C068YBV6KEK

Is there any way to reliably access the params sent in a post-query method like

post_process

or

get_records

? Or would I need to persist those values to instance variables (self.since, self.until etc) on each iteration?

My first thought here would be to use context i.e. set up a parent stream that yields your date range values for

since

and

until

. Then you wouldn't need the paginator above. Related: https://github.com/meltano/sdk/discussions/2880 https://github.com/meltano/sdk/pull/2888

Thanks, will take a look. I think instagram must be paying someone to mess around with their API on a weekly basis.