dejan_pejcev
03/23/2021, 4:24 PMpipelinewise-tap-postgres cannot get messages from the wal log with logical replication?
It just echoes Waiting for first wal message but I see a couple of logs above it says Request wal streaming from 0/FE7F90B8 to 0/FF49E768 (slot pipelinewise_xxxxxxx)douwe_maan
03/23/2021, 4:35 PMdejan_pejcev
03/23/2021, 4:51 PMdouwe_maan
03/23/2021, 4:59 PMpipelinewise-tap-postgres was based on that one, and they both use logical replication in the same way:
• https://github.com/transferwise/pipelinewise-tap-postgres/blob/master/tap_postgres/sync_strategies/logical_replication.py#L609
• https://github.com/singer-io/tap-postgres/blob/master/tap_postgres/sync_strategies/logical_replication.py#L435douwe_maan
03/23/2021, 5:00 PMdoes incremental (key based) sync handles row updates?It does, if you use an
updated_at timestamp as the replication key!dejan_pejcev
03/23/2021, 5:01 PMupdated_at?douwe_maan
03/23/2021, 5:02 PMdouwe_maan
03/23/2021, 5:02 PMupdated_at column whenever a row is updateddouwe_maan
03/23/2021, 5:03 PMdejan_pejcev
03/23/2021, 5:06 PMpipelinewise_xxxxx (database name instead of xxxxx) should be used, it sees that it should stream wal records from LSN X to LSN Y, but receives nul when psycopg2 tries to read from the logdouwe_maan
03/23/2021, 5:17 PMdejan_pejcev
03/23/2021, 5:30 PMdouwe_maan
03/23/2021, 5:36 PMjose_escudero
04/03/2023, 9:50 PMtap-postgres. Were you able to find a solution for this?