Hello, I now try to copy data from Postgres to any...
# singer-taps
p
Hello, I now try to copy data from Postgres to any destination. I am using tap-postgres, default version. How can I filter the data? I don't want to copy all tables in the schema, this would be insane. I want to copy just some elected tables, and just some records. None of the known methods worked for me. This is the meltano.yml:
- name: tap-postgres
variant: meltanolabs
pip_url: git+<https://github.com/MeltanoLabs/tap-postgres.git>
config:
sqlalchemy_url: ****
filter_schemas:
- myschema
metadata:
myschema-MYTABLE:
replication-method: INCREMENTAL
replication-key: KW_UPDATE_DATE
I tried the following: • I tried to set filter_tables parameters but the tap does not allow it. • I tried replication_key_value (thinking I can block copying of particular tables by setting the value in the future) but it got ignored. • I tried to run invoke with parameter -s state.json but that parameter is not allowed • I looked at other variants of tap-postgres but they seem even more concise and not supportive of extra options My problem is that the current version of tap-postgres is copying the entire schema and all the data in all the tables in the schema. This is a no-go... Thank you.
e