GitHub
10/22/2025, 7:02 PMtap | target > state
However for long running targets (1 million records in my current case) it outputs multiple records.
I was under the impression a target would only emit a single state message when it finishes, allowing that state file to be trivially used as the INPUT for the next run of the tap.
It seems my approach would still work using;
tap | target | tail -n 1 > state
Maybe it isn't a bug and is intended; I'm not clear why it logs a variable amount of state messages really, so maybe it is just that the spec needs to be more clear on this (it's not that the target logs every state message, since the tap outputted 120 state messages.
Looks like it just logs one every 30 minutes, as per the log;
"One or more records have exceeded the max age of 30.0 minutes. Draining all sinks."
### Code
No response
meltano/sdkGitHub
10/22/2025, 7:02 PM