<#231 Follow-up from "Resolve "Bug: `get_starting_...
# meltano-repo-activity
g
#231 Follow-up from "Resolve "Bug: `get_starting_timestamp()` and `get_starting_replication_key_value()` should be cached"" Issue created by MeltyBot on <!date^1633633308^{date_short}|2021-10-07T19:01:48Z> Migrated from GitLab: https://gitlab.com/meltano/sdk/-/issues/232 Originally created by @aaronsteers on 2021-10-07 190148 --- The following discussion from !157 should be addressed: • @aaronsteers started a discussion: (+2 comments)
@edgarrmondragon - This is looking great. I just did another review and there's just one remaining change I'd like to request. Can you add explicit removal of the "starting" value's key during the "finalize" step here: https://gitlab.com/meltano/sdk/-/blob/1378ed3ede701d2edde1d96fbd1db09d2971e3c4/singer_sdk/helpers/_state.py#L242
Since this is one of those tokens that's only relevant while the stream is in-flight, we should explicitly remove it at the end of the sync operation. (Which is the function of
finalize_state_progress_markers()
.)
For unsorted streams, I think it may automatically show up under "progress_markers" which gets purged as a whole object during finalization. However, for "normal" sorted streams, we won't have a "progress_markers" dict and we should remove the start value if present.
meltano/sdk