sam_werbalowsky
04/08/2022, 9:17 PMtap-postgres and overriding primary key. There are numerous answers that suggest using the metadata field to override this by setting table-key-properties, view-key-properties or key-properties. I’ve tried all of these with an empty array and it is still being inserted as a primary key into the snowflake table. I have tried:
• deleting the snowflake database and starting from scratch.
• reinstalling the tap and target
• for a sanity check, creating a new tap that inherits from tap-postgres
Looking through the logs for a job I can see that the key_properties in the stream doesn’t get overwritten, even if the catalog is showing the correct metadata. In other words, if I run meltano invoke tap-postgres > output.json, then output.json has key_properties: ["id"] in the SCHEMA json object (This is the actual primary key in postgres, which I want imported just as any other column)
At this point, I’m not sure how else to debug this or where I should look…any ideas? I do see when I run elt that meltano sets the key_properties early in the job, via
2022-04-08T21:16:28.559100Z [debug ] Setting '.streams[53].selected' to 'False'
2022-04-08T21:16:28.559184Z [debug ] Setting '.streams[53].replication_method' to 'INCREMENTAL'
2022-04-08T21:16:28.559272Z [debug ] Setting '.streams[53].replication_key' to 'updated_at'
2022-04-08T21:16:28.559357Z [debug ] Setting '.streams[53].table_key_properties' to '[]'
2022-04-08T21:16:28.559436Z [debug ] Setting '.streams[53].table_key_properties' to '[]'
2022-04-08T21:16:28.559514Z [debug ] Setting '.streams[53].key_properties' to '[]'
but it doesn’t seem like it is actually doing anything.