sam_werbalowsky
04/08/2022, 9:17 PMtap-postgres
and overriding primary key. There are numerous answers that suggest using the metadata field to override this by setting table-key-properties
, view-key-properties
or key-properties
. I’ve tried all of these with an empty array and it is still being inserted as a primary key into the snowflake table. I have tried:
• deleting the snowflake database and starting from scratch.
• reinstalling the tap and target
• for a sanity check, creating a new tap that inherits from tap-postgres
Looking through the logs for a job I can see that the key_properties
in the stream doesn’t get overwritten, even if the catalog is showing the correct metadata. In other words, if I run meltano invoke tap-postgres > output.json
, then output.json
has key_properties: ["id"]
in the SCHEMA json object (This is the actual primary key in postgres, which I want imported just as any other column)
At this point, I’m not sure how else to debug this or where I should look…any ideas? I do see when I run elt
that meltano sets the key_properties
early in the job, via
2022-04-08T21:16:28.559100Z [debug ] Setting '.streams[53].selected' to 'False'
2022-04-08T21:16:28.559184Z [debug ] Setting '.streams[53].replication_method' to 'INCREMENTAL'
2022-04-08T21:16:28.559272Z [debug ] Setting '.streams[53].replication_key' to 'updated_at'
2022-04-08T21:16:28.559357Z [debug ] Setting '.streams[53].table_key_properties' to '[]'
2022-04-08T21:16:28.559436Z [debug ] Setting '.streams[53].table_key_properties' to '[]'
2022-04-08T21:16:28.559514Z [debug ] Setting '.streams[53].key_properties' to '[]'
but it doesn’t seem like it is actually doing anything.