Hey everyone, I have an incremental sorted stream that has a request like

orders?start_modified_at=2021-06-01T12:00:00&page=1

I set the start date parameter in

get_url_params()

by using

self.get_starting_timestamp(context)

. I expected

self.get_starting_timestamp(context)

to be constant and return the initial starting state, but since the stream is sorted, it is emitting the incremental state and

self.get_starting_timestamp(context)

is always progressing. The pagination is then thrown off since I am also changing the

start_modified_at

at the same time as the page number. • Is this expected behavior for

self.get_starting_timestamp(context)

when the stream is sorted? • Is there another way to get the initial starting state?

interesting! I may be a bit naïve here so forgive me, but I would think in your case the state shouldn't update until all pages have been read from? I'd guess there's some way to tell the sdk not to update your state

Yea, on one hand, I want the state to update so if the job is interrupted, it can resume from that updated start_modified_at on the next run. But I don't want

self.get_starting_timestamp(context)

/ start_modified_at to update midrun on each call.

Got it, so every record that you're reading in here has a

modified_at

If it fails, I want it to follow this behavior (which it is correctly doing).

Is this expected behavior for 

self.get_starting_timestamp(context)

 when the stream is sorted?

No, this sounds like a bug to me. I think get_starting_timestamp() should be cached. I'm opening an issue...