I'm trying to set up multiple pipelines where a bu...
# plugins-general
m
I'm trying to set up multiple pipelines where a bunch of extractors point to the same loader, but I want to have different configs for the loaders. I see pipeline-specific-configuration in the docs, and I see that you can define your environment variables within the
meltano.yml
file, but I don't want to actually store the values there and would rather have them within the
.env
file.
d
@michael_cooper You can't currently have multiple different values for the same plugin+setting combination in
.env
, unfortunately. This is something that configuration profiles would make possible (https://gitlab.com/meltano/meltano/-/issues/2294), but I don't expect to get to that for at least another month.
Your best bet right now would be to add the same loader to your project as a custom plugin multiple times, which a different
name
each time, so that you get unique setting env vars. Would that work?
m
Yeah, that would work. Thank you!
Does it install the code multiple times then?
d
It does, yeah
Which is what makes it a workaround, not a full solution 🙂
r
@stefano_nicolai this could be this could be helpful