Hi, I'm getting an issue in tap-postgres where I can't actually exclude columns from tap-postgres using either the

!schema-tablename.column

or explicitly selecting <only> the columns I want: exclude case:

select:
  - '!schema-tablename.badcolumn

explicit select case:

select:
  - schema-tablename.col1
  - schema-tablename.col2
  # leaving out the badcolumn

When I run

meltano select tap-postgres --list

it shows only the selected columns. But when I actually run tap-postgres, it still selects all columns. Anyone have experience with this? Note this is running with the pipelinewise/transferwise variant.

Hey @Russell Romney! Can you try the workaround described in https://github.com/meltano/meltano/issues/2848#issue-1252464595? ie,

rm .meltano/run/tap-postgres/*

@Edgar Ramírez (Arch.dev) thanks - this did not remove the fields from the select statement I'm confirming that in .meltano/run/tap-postgres the field, let's call it "fieldname", is excluded in the regenerated run plan,

{
          "breadcrumb": [
            "properties",
            "FIELDNAME"
          ],
          "metadata": {
            "sql-datatype": "jsonb",
            "inclusion": "available",
            "selected-by-default": true,
            "selected": false
          }
        },

Ah, that variant may just not support column selection: https://github.com/transferwise/pipelinewise-tap-postgres/issues/200

Ah. Thanks for the help there, I hadn't found that issue yet in my searching