Hi meltano'ers, I have a tap-postgres setup with a log_based replication method. We ran into an issue that the extractor suddenly stopped fetching newer records after a certain amount of time. Here's what we have tried so far: 1. Cleared the state. However, state gets back where it was before clearing 2. Confirmed the wal keep size to be -1, meaning no limits on wal file size 3. Advanced pg confirmed_flush_lsn to the current wal's lsn. However, whenever meltano starts, pg reverts back to the lsn where meltano got stuck 4. Dropped the replication slot and recreated it again. This deleted wal files between the meltano lsn and current wal's lsn so that we had to recover the gap manually. The tap is now running normally since then We ran out of ideas to resolve this issue efficiently without dropping the replication slot every now and then. Any feedback is very much appreciated. THANK YOU

Do you have any logs when it'd stop pulling data? Can you share

@visch Attached are two logs files. Dec7th is the one where we noticed that no data being extracted (keeps loading one record only for every run), and Dec9th is when we resolved the issue by dropping the replication slot and recreating it again. The job runs in a 10 minutes intervals

I don't have the time to deep dive this, but maybe someone can logs are helpful

the same issue happened again where the meltano runs bring no data. does anyone have any idea how to let meltano proceed without dropping the replication slot and recreating it again