If I need to do a complete refresh for a pipeline using

meltano run --full-refresh

and the tap fails midway through, will the next run of

meltano run

for the same

tap

/

target

use state from the

--full-refresh

run or could it potentially pick up stale state from another

meltano run

? For example:

meltano run --full-refresh some-tap some-mapper some-target
meltano run some-tap some-mapper some-target <- will this use state created by the previous line?

Mostly wondering because I just ran a

--full-refresh

that failed after a couple hours, then the subsequent run without a

--full-refresh

finished in 15 minutes but I'm clearly missing tons of data in the target.

In my experience it depends on the tap you use. Most of them are saving state during processing and in case of unexpected interruption would continue from the last saved state.