I have a stream with the following params:

def get_url_params(self, context: Optional[dict], next_page_token: Optional[Any]) -> Dict[str, Any]:
        """Return a dictionary of values to be used in URL parameterization."""
        params: dict = {}
        params["per_page"] = self.config.get("per_page")
        params["sort_by"] = "date_modified"
        params["sort_order"] = "ASC"
        params["start_modified_at"] = self.get_starting_timestamp(context)
        params["page"]=next_page_token
        return params

Now that stream is clearly sorted, but if I set

is_sorted = True

,

self.get_starting_timestamp(context)

updates after every call and messes up the pagination. The easy solution is to not set

is_sorted = True

, but I like that it will continue if interrupted. Any suggestions or am I missing something?

@dan_ladd I think the SDK should be caching that initial value and this sounds like a bug. Perhaps our test cases were too skewed towards the initial call or unsorted streams, but

get_starting_timestamp()

should not change during the course of the sync - Do you mind opening an issue to describe what you are seeing? As a workaround, you could cache the result yourself in a custom property at the beginning of the stream sync. That's not ideal though, and I believe the expected behavior should be that it returns a single value through the lifecycle of the sync.

Ok yea that makes sense to cache it at the start as a workaround. I will play around with it a little and open an issue to confirm it. Thanks AJ!

Thanks, @dan_ladd. Much appreciated. 👍

Ah seems an important note here is it's only happening with a child stream

Logged for tracking: https://gitlab.com/meltano/sdk/-/issues/208

oh wow didn't realize i double posted the same problem. I think I'm losing it... Thanks for opening up that issue AJ and confirming