I'm ingesting data from a postgresql db using melt...
# troubleshooting
d
I'm ingesting data from a postgresql db using meltanolabs tap-postgres. I'm using meltano-map-transform to remove some sensitive fields. As I understand it, the tap reads the fields, then meltano-map-transform filters them out. I'd like to, instead, tell the tap to not read these fields. I know I can do this using a view in the source db, but that's cumbersome. Is there any other way of doing this?
1
r
You probably want to use
meltano select
to deselect the sensitive fields:
Copy code
meltano select tap-postgres --exclude my_table.my_sensitive_field
meltano select tap-postgres --all
which will add the following your your `meltano.yml`:
Copy code
select:
    - '!my_table.my_sensitive_field.*'
    - '*.*'
https://docs.meltano.com/concepts/plugins#select-extra
👍 1
d
Thank you!
np 1