If I need to do a complete refresh for a pipeline using

meltano run --full-refresh

and the tap fails midway through, will the next run of

meltano run

for the same

tap

/

target

use state from the

--full-refresh

run or could it potentially pick up stale state from another

meltano run

? For example:

meltano run --full-refresh some-tap some-mapper some-target
meltano run some-tap some-mapper some-target <- will this use state created by the previous line?

Mostly wondering because I just ran a

--full-refresh

that failed after a couple hours, then the subsequent run without a

--full-refresh

finished in 15 minutes but I'm clearly missing tons of data in the target.

AFAIK even if --full-refresh fails, it persists the state regularly during the execution, so if you execute it after the fail, without --full-refresh, it should be capable of utilizing the state from the previous execution...

Thanks, something odd must be happening with the tap so I'll just need to keep trying