Hi I have a question about custom tap development. I am currently using a custom tap hosted on an internal private repository. Why is it that for a custom tap, I must define

namespace

,

settings

, etc. Whereas with something like

tap-postgres

, i simply have to install the plugin and then I do not have to define these things. I have configured my

config_jsonschema

in my custom taps

tap.py

, but this does not seem to get picked up by meltano.

I can answer the first question. Meltano uses this discovery.yml file for known taps so you don't need to do that effort, look here for tap-postgres for example: https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/bundle/discovery.yml#L1557

https://docs.meltano.com/tutorials/custom-extractor#add-the-plugin-to-your-meltano-project have you tried this @elliot

@visch I have! And the tap works as expected (once I had configured these on my meltano.yml). However, I was asking why I needed to do this with a custom tap.

Taps don't automatically populate meltanos "discovery" feature 😕

Thanks @visch I appreciate the help 🙂

Sorry i don't know what you mean with the config_jsonschema

This is an example: https://gitlab.com/autoidm/autoidm-tap-bamboohr/-/blob/master/tap_bamboohr/tap.py#L26

It's a good question, maybe if someone from the Meltano team can check it out they can tell us 👀

Hi @elliot! The reason is, singer packages don't usually communicate their settings in a programmatic way Meltano can use, so historically those settings had to be specified either in

meltano.yml

if they're custom or submitted to MeltanoHub. So, for example the settings of

tap-postgres

are defined upstream and then pulled into your project as a lockfile, and that is why you don't need to specify its settings yourself. That said, it would be ideal if taps, or at least those built with the SDK, could communicate their settings and other metadata directly to Meltano: https://github.com/meltano/sdk/issues/377