Hey all I m reading the singer docs on <https github com sin Meltano #singer-tap-development

Hey all, I'm reading the singer docs on <discovery...

niall_woodward

01/26/2022, 5:06 PM

Hey all, I'm reading the singer docs on discovery mode and it says that

A tap is free to write ANY type of metadata they feel is useful for describing fields in the schema

I can't seem to find an SDK built-in way of adding additional key and values to the metadata returned from --discover. Does anyone have insight on this?

niall_woodward

01/26/2022, 5:07 PM

For instance, a stream entry returned from my tap is:

Copy code

{
  "tap_stream_id": "report_jobs_datadownload_report_id_590",
  "replication_method": "FULL_TABLE",
  "key_properties": [],
  "schema": {
    "properties": {},
    "type": "object"
  },
  "stream": "report_jobs_datadownload_report_id_590",
  "metadata": [
    {
      "breadcrumb": [],
      "metadata": {
        "inclusion": "available",
        "selected": true,
        "table-key-properties": []
      }
    }
  ]
}

I'd love to be able to add a 'name' attribute under metadata so that a user can see both the ID and the name when selecting the stream

edgar_ramirez_mondragon

01/26/2022, 6:04 PM

@niall_woodward something like https://gitlab.com/meltano/sdk/-/issues/211?

aaronsteers

01/26/2022, 6:04 PM

(Sorry - deleted my last post because I was confused on catalog vs state as the use case.)

niall_woodward

01/26/2022, 6:49 PM

Ah yes, that @edgar_ramirez_mondragon! Will drop a thumbs up

edgar_ramirez_mondragon

01/26/2022, 6:52 PM

Thanks @niall_woodward if you could add a code example of how the final catalog entry or metadata would like for your use case would be super useful 😄

niall_woodward

01/26/2022, 7:02 PM

https://gitlab.com/meltano/sdk/-/issues/211#note_822627687

Open in Slack

Previous Next