https://meltano.com/ logo
#announcements
Title
# announcements
h

high-oil-4073

03/23/2021, 4:24 PM
Does anybody know why
pipelinewise-tap-postgres
cannot get messages from the wal log with logical replication? It just echoes
Waiting for first wal message
but I see a couple of logs above it says
Request wal streaming from 0/FE7F90B8 to 0/FF49E768 (slot pipelinewise_xxxxxxx)
r

ripe-musician-59933

03/23/2021, 4:35 PM
h

high-oil-4073

03/23/2021, 4:51 PM
initially it was working, for 2-3 days, and stopped all of a sudden :
r

ripe-musician-59933

03/23/2021, 4:59 PM
does incremental (key based) sync handles row updates?
It does, if you use an
updated_at
timestamp as the replication key!
h

high-oil-4073

03/23/2021, 5:01 PM
meaning my apps should take care of the
updated_at
?
r

ripe-musician-59933

03/23/2021, 5:02 PM
yeah, key-based replication will only pick up updates if it has a way to tell if a row got updated since the last sync
It's pretty common for ORM frameworks (like ActiveRecord in Rails) to automatically increment an
updated_at
column whenever a row is updated
I have no idea why the WAL approach would suddenly break though 😕 That seems like something to dive into deeper. It doesn't look like the tap is using it incorrectly
h

high-oil-4073

03/23/2021, 5:06 PM
the tap detects everything correctly. It sees it needs to use the
pipelinewise_xxxxx
(database name instead of xxxxx) should be used, it sees that it should stream wal records from LSN X to LSN Y, but receives nul when psycopg2 tries to read from the log
r

ripe-musician-59933

03/23/2021, 5:17 PM
Right. I'm sure that that tap is running in production with log-based replication in various places, so I wonder if there's something off about your postgres server
h

high-oil-4073

03/23/2021, 5:30 PM
hmmm I’m using AWS RDS Aurora Postgres v12.4
r

ripe-musician-59933

03/23/2021, 5:36 PM
Hmm, no reason to think that wouldn't be configured perfectly then
c

cuddly-insurance-63176

04/03/2023, 9:50 PM
@high-oil-4073 finding a similar issue with
tap-postgres
. Were you able to find a solution for this?