@douwe_maan I think storing all the state messages outputted by a tap during a run will be valuable. https://github.com/singer-io/getting-started/blob/master/docs/SPEC.md#state-message these are structured logs and can be ingested in meltano db or some other store for future use. 1 use case can be to reduce the pain of cold start problem

@nehil_jain We're already storing that state to allow picking up where a previous run left off when

--job_id

is provided to

meltano elt

; are you suggesting something beyond that? 🙂

yeah i am suggesting storing it during an in progress run and not just the last message

Aah, right, store it even if the job failed, because something may have made it into the target. That makes sense, and shouldn't be too hard to implement!

you got it