Hello I have some problems while building my custom tap the Meltano #meltano-plugin-development

Hello, I have some problems while building my cust...

luis_ventura

01/06/2023, 4:36 PM

Hello, I have some problems while building my custom tap, the settings are not exposed when I install the tap in another environment (https://meltano.slack.com/archives/C01TCRBBJD7/p1672767927246459). I don’t know if it is linked but in this environment, I have also a problem with a child stream that sync all data even if I don’t use the full refresh option. Is there any thing I’m missing to correctly expose the tap settings ?

pat_nadolny

01/06/2023, 4:56 PM

Hey @luis_ventura - yeah right now for custom plugins you need to define your settings explicitly in your meltano.yml in order for them to be accessible. Meltano doesnt pull the settings out of your custom extractor automatically as of today.

pat_nadolny

01/06/2023, 5:00 PM

If the context is helpful - when you add a tap listed on the hub it pulls the settings down into your

plugins

directory as a lock file, then meltano knows to reference those when doing something like

meltano config tap-csv list

. For custom plugins that settings lock file doesnt exist so meltano doesnt know the available settings, so it just falls back to whats set in your meltano.yml. In the future a nice feature would be to have it fall back to calling the tap's

--about

command (assuming its SDK based) to get available settings.

luis_ventura

01/06/2023, 5:10 PM

Hello @pat_nadolny thanks for the clarification 👌

pat_nadolny

01/06/2023, 5:17 PM

I also opened a feature request for this since I couldnt find one https://github.com/meltano/meltano/issues/7156

Open in Slack

Previous Next