Hello hello! Happy Wednesday 😄 I had a question regarding starting timestamp & replication methods. I was wondering how having an incremental vs full table replication method influenced the starting timestamp. I'm having issues where a stream which WAS incremental is now full table, but it still seems to take into account state. I believe this is because the

get_starting_timestamp

function only retrieves the state from the context while disregarding the replication method, while the replication method only affects if the replication signpost is stored. Therefore, a stream which has gone from incremental to full table would need to get it's state wiped in order to function properly with the

get_starting_timestamp

function. @edgar_ramirez_mondragon would you be able to confirm?

In short, yes. A full-table stream can use

get_starting_timestamp

will use state regardless of the replication method. I'm curious if you changed the replication method in-code, or via the catalog? If the former, then you should be able to remove calls to

get_starting_timestamp

so state is no longer used for that stream. If the latter, then you have to manually remove the stream's bookmark from the state. Also, it's arguably a bug as is but I'm not sure if we should make

get_starting_timestamp

a no-op (with a warning logged) if the replication method is not full-table.

I change the replication method in the metadata (catalog) of my main project (not in the tap). I wanted to be able to have 2 different streams which use the same base stream but with different replication methods. Currently, my solution is simply to create a new stream which won't have bookmark 😅 , but in the future I'll need to to find a way to get & modify state in production, probably through an

ECS exec

operation (we're on ECS fargate)

Gotcha. Do log an issue in the SDK repo for making

get_starting_timestamp

always return null values for full-table streams 🙂

Thanks @edgar_ramirez_mondragon!