I am using Meltano for ETL to extract data from a source PostgreSQL database using

tap-postgres (MeltanoLabs variant)

and load it into a destination PostgreSQL database using

target-postgres (TransferWise variant)

. However, I have noticed that when I delete rows from the source PostgreSQL database, the deleted rows are not removed from the destination PostgreSQL database. I have tried: • Using the`hard_delete=true` option of the target-postgres(https://hub.meltano.com/loaders/target-postgres--transferwise/#hard_delete-setting), but this does not seem to delete the rows from the destination database. Here is the command I am using to perform the ETL:

meltano run tap-postgresql target-postgres

I have verified that the rows have been deleted from the source PostgreSQL database by running a SQL query, but the deleted rows still exist in the destination PostgreSQL database after running the job. Is there a way to hard delete in destination? Any help or suggestions would be greatly appreciated. Thank you.

@vuonglv does the Meltano SDK-based tap support log-based replication? I'm not sure it does. That's the only way to have deletes propagated to the target. (Except full_table mode, anyway, but that's not especially scalable...)

You’re correct @thomas_briggs, the MeltanoLabs variant of the SDK doesn’t support log-based replication yet, so deletes are not propagated via the

_SDC_DELETED_AT

column

@edgar_ramirez_mondragon so, how tap-postgres can detect a row is deleted and mark

_SDC_DELETED_AT

value? How can I config tap-postgres(any variants) to support this requirements? Thanks for your support!

There’s some setup on the database side, then you’ll have to set the replication-method to LOG_BASED.