Hi all, I have an issue using select config in my ...
# troubleshooting
s
Hi all, I have an issue using select config in my tag-postgres, and want to know how to fix it. I have a source table table_a in one postgresql database with column a, b, c. I want to only load a, b from table_a to another table table_b in another postgresql database. - name: tap-postgres variant: transferwise pip_url: pipelinewise-tap-postgres config: host: localhost user: postgres dbname: local_db filter_schemas: public default_replication_method: FULL_TABLE select: - public-table.a - public-table.b When I ran the command
meltano run tap-postgres target-postgres --full-refresh
, for some reason, the column c was added to table_b. is this normal behavior? or is a bug? another issue, when I update table_a, run
meltano select  tap-postgres --list
, the new columns are not showing up, how to fix this issue?
s
@sean_han So you ran your pipeline with an without the --full-refresh flag? If you do it without full refresh the column c isn't added, if you use --full-refresh, the column is added? Is that correct? Does sound odd. Could you reproduce that? And for the select statement, the select --list lists selected attributes, and you updated the select statement? Or did you update the table itself? Did you run --list --all?
s
@Sven Balnojan thanks for help. I ran using '--full-refresh' flag, and column c was added. for second issue, after I added a column to my source table, both --list or --list --all won't list the new column.
even I added this column to select patterns