Hello guys,
Has anybody configured the pipelinewis...
# plugins-generald
dejan_pejcev
02/05/2021, 4:27 PMHello guys,
Has anybody configured the pipelinewise tap postgres to use logical replication (and read from the replication log) through Meltano? I am having issues
d
douwe_maan
02/05/2021, 4:29 PMI know that @matt_cooley used pipelinewise-tap-mysql with LOG_BASED replication: https://meltano.slack.com/archives/C013EKWA2Q1/p1608097828264300 Presumably tap-postgres would behave similarly
m
matt_cooley
02/05/2021, 4:37 PMHi! I did in fact get it set up. @dejan_pejcev What issues are you having?
matt_cooley
02/05/2021, 4:38 PMOh, actually, I was doing a
tap-mysql → target-postgresd
dejan_pejcev
02/05/2021, 4:38 PMLooks like the pipelinewise postgres tap doesn’t read from the wal2json replication slot.
It sees correctly that it should use LOG_BASED, and for each table it always does logical_initial, which is the initial full sync, and after that it should read changes from the replication log, but it does again logical_initial
m
matt_cooley
02/05/2021, 5:11 PMYeah that’s different than the issues I was having and its a different tap. I actually ended up using a Singer Tap. But they’re not as well maintained so it’s not a super ideal solution.
d
dejan_pejcev
02/05/2021, 6:46 PMUPDATE:
I was missing the
--job_id5 Views