I have a situation where it took longer than I expected to replicate a mysql database and now I have some streams that are missing their binary logs:

meltano | Extraction failed (1): Exception: Unable to replicate binlog stream because the following binary log(s) no longer exist: mysql-bin-changelog.702290

I see there’s an option to use the --full-refresh command to “clear the state.” My question is as follows: If I wanted to clear the state of specific streams (and not all streams) for a particular id, does the --full-refresh command work in combination with the the --select command so that I can retain the state for some streams?

Hi @deactivateduser278489!

If I wanted to clear the state of specific streams (and not all streams) for a particular id, does the --full-refresh command work in combination with the the --select command so that I can retain the state for some streams?

Unfortunately no. At the moment Meltano has no visibility into the state of individual streams and treats the state object as a single entity so it's not possible to selectively do the full refresh. That said, we've been looking into how that might work and discussing with the broader singer.io ecosystem to settle on some standards: https://gitlab.com/meltano/meltano/-/issues/2903, https://github.com/MeltanoLabs/Singer-Working-Group/pull/23.