Hello all. I'm working on a meltano project that involves a number of

--custom

taps and targets (github, redshift, etc). I want to leverage Meltano's data integration functionality through the

meltano etl

command with these custom plugins, but the plugins expect their configuration as .json files with a path specified in the invocation of the plugin. For example, this works (cutting meltano out of the loop): `

~/.virtualenvs/tap-something/bin/tap-something \
  | ~/.virtualenvs/target-redshift/bin/target-redshift \
    --config ~/singer.io/target_redshift_config.json

Is there any way to make these plugins play nice with the

meltano etl

command as part of a data integration pipeline? I've tried setting environment variables for these configurations, but the plugins don't seem to like that. Any help appreciated :)

@dry-city-1354 Meltano knows that Singer taps and targets expect a

--config

option pointing at a

config.json

file, so it will automatically generate the files and invoke the executables as appropriate when you run

meltano elt

. You can see the exact commands that are executed by

meltano elt

when you enable debug mode:

meltano --log-level=debug elt ...

You can see the configuration that is passed to the plugin when you run

meltano config <plugin>

. Does that show the correct configuration for your tap and target?

I'll take a look and report back. Thanks for the help!

@dry-city-1354 Thanks for the update, glad you figured it out!