Hi everyone, We need your help 🙂 Currently we use a mysql-tap in production with binlog replication. We schedule to migrate our mysql database towards a new instance and so a new replication. The big impact is the binlog replication start at new point and the current binlog stream doesn't exist in the new instance. For instance, if we run mysql-tap on the new instance we raise this following error:

Exception: Unable to replicate binlog stream because the following binary log(s) no longer exist: mysql-bin-changelog.252519

Do you know a process to handle this case ? Or the only option is to make a full refresh and restart the binlog replication from the new instance ?

Theoretically, I think it should be possible to adjust all the binlog positions in the meltano database to reflect the positions in the new DB's binlog. I would expect that to be tedious and error-prone though so personally I wouldn't attempt it. It's probably the same amount of work just to re-initialize the whole pipeline. 😕

I considered the first option, as you said I think there are many possible sources of error. Even if it works, it is possible to lose information and so on re-initialize data. it could be a good way to improve meltano in nexts months.

@alexandre_brehelin - As of now, I think your assessment is basically correct and I agree with @thomas_briggs. You could edit the state info - but the process is error prone. If it helps, we do have some new-ish

meltano state

CLI commands that could help with 'get' / 'set' / 'copy' operations - so you don't have to touch meltano state directly. (Also, this makes it easy for you to backlog your old state before making changes.)