I'm trying to set

key_properties

on a tap but the target (target-redshift) won't recognise it at all. Keep getting the an error from the target saying the

key_properties

is not set. I'm using the tap-apple-search-ads (forked with additions). Here's what my meltano.yml looks like, under the plugin -> tap-apple-search-ads:

metadata:
      impression_share_reports:
        replication-method: INCREMENTAL
        replication-key: date
        key-properties: [date]

When I run

meltano invoke

and do a dump of the catalog, it does correctly specified metadata to generated catalog. But not sure if this same catalog is being used when I use

meltano run

Try to dump the data and see if key properties are set properly in your schema. So

meltano invoke tap-name > out

Yep you're right, not in the stream:

{
  "type": "SCHEMA",
  "stream": "impression_share_reports",
  "schema": {
    "properties": {
      "date": {
        "format": "date",
        "type": "string"
      },
      "appName": {
        "type": "string"
      },
      "adamId": {
        "type": "string"
      },
      "countryOrRegion": {
        "type": "string"
      },
      "searchTerm": {
        "type": "string"
      },
      "lowImpressionShare": {
        "minimum": 0,
        "maximum": 1,
        "type": "number"
      },
      "highImpressionShare": {
        "minimum": 0,
        "maximum": 1,
        "type": "number"
      },
      "rank": {},
      "searchPopularity": {
        "type": "integer"
      }
    },
    "type": "object"
  },
  "key_properties": []
}

Tap doesn't support key properties passed in via the catalog it seems. If you think it should work, then you could add them with a mapper https://sdk.meltano.com/en/latest/stream_maps.html#unset-or-modify-the-stream-s-primary-key-behavior 🤷

I've forked the tap and modified the writing of the schema message to get the key_properties from metadata. That has solved the issue. Thank you.