hey just a little interesting scenario i’ve noticed when messing with states. When specifying the following:

name = "wfmhistoricaladherence-raw" 
primary_keys = ["$.userId"]
replication_key = "endDate"

I’ve noticed that calling the expected replication key methods, i get: •

self.get_starting_timestamp(self)

-> the config.start_date value, even if I have a datetime/ISO replication_key (2010-01-01T000000+00:00) • and

self.get_starting_replication_key_value(self.stream_state)

-> None But when I call

self.stream_state["replication_key_value"]

I get the replication value i’m after.

tap-genesyscloud | time=2021-10-07 23:20:58 name=tap-genesyscloud level=INFO message=self.stream_state: {'replication_key': 'endDate', 'replication_key_value': '2021-10-07T12:05:44.311Z'}
tap-genesyscloud | time=2021-10-07 23:20:58 name=tap-genesyscloud level=INFO message=self.get_starting_replication_key_value from self: None
tap-genesyscloud | time=2021-10-07 23:20:58 name=tap-genesyscloud level=INFO message=self.get_starting_replication_key_value frin stream_state: None
tap-genesyscloud | time=2021-10-07 23:20:58 name=tap-genesyscloud level=INFO message=2010-01-01T00:00:00+00:00

Am I approaching this incorrectly? Been scratching my head around this for a few hours. • Using the Meltano SDK to initialise the tap project • calling these within

get_records()

so I can make an appropriate api call out • No changes to the tap related to state. Just the vanilla stuff in the base Stream class

When you call

get_starting_timestamp

you should also pass context along with it. That is how Meltano stores the context about this specific Stream "run"

Thanks Derek! This is super super helpful man

Yeah no problem instead of calling timestamp directly, try using

get_starting_replication_key_value

first with context provided. That might get you there