I am using target-postgres to write to a table. That works great for a given job. I then added a generated column to the destination table that I want Meltano to ignore entirely. However, my

INSERT

statements are now include the new column, which PostgreSQL doesn't like. (The new column does not appear in the schema generated by

meltano invoke <my tap> --discover

, which is correct.) Is there a way to tell Meltano/target-postgres to only include in its generated SQL columns that appear in the Meltano schema? Thanks in advance!

can you share your meltano.yml file? I can tell you that I have columns that Meltano doesn't touch in Postgres with the meltano labs target-postgres relatively frequently I'm 95% confident

Unfortunately the file is pretty large and contains some semi-sensitive information. But here are the loader and extractor:

version: 1
default_environment: dev
project_id: <obfuscated>
plugins:      
    extractors:
        - name: tap-customer-spreadsheets-anywhere
                inherit_from: tap-spreadsheets-anywhere
                variant: ets
                pip_url: git+<https://github.com/><my_fork>/tap-spreadsheets-anywhere.git
                config:
                    tables:
                        - path: sftp://<obfuscated>
                            name: my_table
                            pattern: .*\.xlsx
                            start_date: "2025-04-10T00:00:00Z"
                            key_properties:
                                - user_id
                                - group_id
                            schema_overrides:
                                user_id:
                                    type: ["null", string]
                            format: detect
                            universal_newlines: false
                            sample_rate: 10
                            max_sampling_read: 2000
                            max_sampled_files: 3
                            prefer_number_vs_integer: true
                            selected: true                
    loaders:
        - name: target-postgres
            variant: meltanolabs
            pip_url: git+<https://github.com/healthrosetta/target-postgres.git>
            config:
                database: mydb
                default_target_schema: dest
                host: <obfuscated>
                port: 5432
                user: meltano
                stream_maps:
                    this_is_the_relevant_stream:
                        source_file_date: __NULL__