Is there a way to use `state` to decide where to s...
# troubleshooting
a
Is there a way to use
state
to decide where to start querying a tap via timestamp filter, but NOT write the updated state at the end of a
meltano run
back to the state store? Reasoning: I use a Azure container for state storage in test/prod. In local dev, I sometimes want to point to the existing state marker in test/prod as a starting point to reference new data, rather than a local run that brings in tons of data. But I don't want to update the state as that would interrupt my test/prod processes. That way I can iterate on the same small dataset each time. Basically I would like the option to treat the state as immutable/read only.
v
If my tap supported
start_date
I'd try to use that as it's easier than manipulating state directly and it's made for what you're trying to do right now. If you're really set on using
state
directly then maybe something like
meltano state get
to get your production state, pipe that to a file, then use
meltano state set
to store the state from production to your dev local side.
a
Good tip, thanks.