dan_ladd
08/19/2021, 7:07 PMdef get_url_params(self, context: Optional[dict], next_page_token: Optional[Any]) -> Dict[str, Any]:
"""Return a dictionary of values to be used in URL parameterization."""
params: dict = {}
params["per_page"] = self.config.get("per_page")
params["sort_by"] = "date_modified"
params["sort_order"] = "ASC"
params["start_modified_at"] = self.get_starting_timestamp(context)
params["page"]=next_page_token
return params
Now that stream is clearly sorted, but if I set is_sorted = True
, self.get_starting_timestamp(context)
updates after every call and messes up the pagination.
The easy solution is to not set is_sorted = True
, but I like that it will continue if interrupted. Any suggestions or am I missing something?