Hey ya’ll, running into an issue with tap-stripe that I’m hoping some meltano experts can shed some light on. Background: tap-stripe (singer-io) does this weird thing… where if it detects that it can’t process a stream (due to some api restrictions) it falls back by DELETING a state entry element, writing a new state message, then crashing with an exception. It expects that the state in the database will be updated. On the next run, the modified state will trigger a different path, resolving the api restriction for the stream. However, this doesn’t seem to be the case with the meltano state db. I see the state message it writes before crashing, but that’s not what’s saved in the database. The debug logs looks like 1.

[debug    ] {"type": "STATE", "value": .....} cmd_type=extractor name=tap-stripe (out) run_id=9833f899-f493-4a2f-8f84-f12030ea20de state_id=stripe stdio=stdout

2.

[info     ] CRITICAL Provided current bookmark date for event updates is older than 30 days. Hence, resetting the bookmark date of respective charges/None stream to start date. cmd_type=extractor name=tap-stripe run_id=9833f899-f493-4a2f-8f84-f12030ea20de state_id=stripe stdio=stderr

3.

[debug    ] {"bookmarks": { ... NOT STATE FROM STEP (1) BUT OLDER VERSION ... } cmd_type=loader name=target-duckdb (out) run_id=9833f899-f493-4a2f-8f84-f12030ea20de state_id=stripe stdio=stdout

4.

[debug    ] Added to state stripe state payload {'singer_state': {'bookmarks': ...SAME AS 3... }}

5.

[info     ] Incremental state has been updated at 2024-03-20 09:25:27.099863.

So it seems like the meltano singer-io tap executor is not saving the state tap-stripe’s expecting. I noticed in (3) that the name of this bookmark was from my target-duckdb, which suggests that maybe somehow meltano’s reading the state from the target, and it’s not seeing the new state that the tap is writing before crashing out? The target here is also a singer-io target-duckdb.

suggests that maybe somehow meltano’s reading the state from the target

The target emits the last state (the actual value, not a

STATE

message) it received that it processed records for in step 3, so that's why you see an older state there. IMO it's the target responsibility to decide if after a crash it should emit the very last state it received, or the one that it deemed correct to avoid data loss. You can probably confirm this by running a stepwise singer pipeline:

meltano invoke tap-stripe > tap-stripe.singer.jsonl
cat tap-stripe.singer.jsonl | meltano invoke target-duckdb

Thanks Edgar! Sounds like I need to patch the target so it can handle the last state message