https://linen.dev logo
#singer-tap-development
Title
# singer-tap-development
d

dan_ladd

08/19/2021, 7:07 PM
I have a stream with the following params:
Copy code
def get_url_params(self, context: Optional[dict], next_page_token: Optional[Any]) -> Dict[str, Any]:
        """Return a dictionary of values to be used in URL parameterization."""
        params: dict = {}
        params["per_page"] = self.config.get("per_page")
        params["sort_by"] = "date_modified"
        params["sort_order"] = "ASC"
        params["start_modified_at"] = self.get_starting_timestamp(context)
        params["page"]=next_page_token
        return params
Now that stream is clearly sorted, but if I set
is_sorted = True
,
self.get_starting_timestamp(context)
updates after every call and messes up the pagination. The easy solution is to not set
is_sorted = True
, but I like that it will continue if interrupted. Any suggestions or am I missing something?