What's the best way to go about ensuring a tap emits state at regular intervals with a timestamp-based replication key? Is this already handled by the SDK based on record count?

Yeah, pretty much already handled by the SDK. Taps emit a STATE message every 10,000 records by defaults (source), and targets emit state to stdout either based on record count or record age.

Right, thanks - so I would just need to define

STATE_MSG_FREQUENCY

in my stream class if I wanted to tweak that behaviour? Tossing up whether that is suitable or the timestamp replication key period requirement would still be necessary... I think they both fundamentally achieve the same thing, so probably not.

or the timestamp replication key period requirement would still be necessary

I guess I'm not following what this means

For example, with a start date of

2025-01-01

I want state to be emitted every week's worth of data:

2025-01-08

,

2025-01-15

,

2025-01-22

, etc

Ok, so perhaps the SDK stream classes could have a

should_emit_state

method that developers can override for such use cases. Wdyt?

Yes, that would be ideal I suppose - I can put together an feature request issue later.