Hello, How Meltano support failure internally? Let...
# troubleshooting
c
Hello, How Meltano support failure internally? Let's say I'm backfilling the data and It take 1 day to complete it. If process stops somehow then how Meltano resume the process ?
m
The replication methods,
INCREMENTAL
and
LOG_BASED
both produce
state
messages which meltano can use to hold information about the progress of loading. If your process fails during a sync operation using these methods then Meltano will use the state to resume loading. Read more here: https://docs.meltano.com/concepts/state_backends
c
Yes I saw that but It only write the state at the end of the process
m
Some taps write state messages periodically during the process
c
Do you have any example ?
m
We use tap-oracle, tap-db2, tap-mssql * all of which periodically write state messages and we have encountered failures and resumed successfully