hey guys! is there a provision for supplying meltano.yml file dynamically to

meltano run

command ? I have a use case where I have multiple configurations and I want to specify the path for every file sequentially

I want to specify the path for every file sequentially

Can you expand on this a bit more? Do you mean you want to use multiple yaml files with a single run, or use a different file in each run?

@edgar_ramirez_mondragon I want to use different yaml files in each run.

@silverbullet1 Could you tell us more about this use case? Odds are you either want: • Dynamic config with a single Meltano project (i.e. a single

meltano.yml

file) using inherits_from to have different plugin configurations for the same plugins, and possibly multiple environments to switch between plugin configs and environment variables using the

--environment

CLI option for

meltano run

. • Multiple separate Meltano projects (i.e. multiple

meltano.yml

files, possibly still located within the same repository).

@Will Da Silva (Arch) Yes its more like 2 but I have a generic pipeline(project) which copies data from source to destination, and I have multiple

meltano.yml

files in the same repo. I want to invoke

meltano run my-tap my-target

and optionally specify the path of my meltano.yml file(it might be in some other folder) with

-c /some/other/foldermeltano.yml

@silverbullet1 What is special about your use-case that 2

meltano.yml

files are required? Why is plugin inheritance and multiple environments not sufficient for running your distinct pipelines using a single

meltano.yml

file? In any case, if the

meltano.yml

files are in different directories, you can use the top-level

--cwd

CLI option to specify the directory Meltano should run in, and by extension, which

meltano.yml

file it should use.

@Will Da Silva (Arch) I have different inputs basically. Say team A wants to replicate tables t1, t2 and t3 and team B wants to replicate v1, v2 and v3. I wanted to have the same base Dockerfile to enable this scenario.

--cwd

might work, let me check

@silverbullet1 I strongly recommend using

inherits_from

and

environments

to address this use-case. That is exactly what they were designed to support. Using multiple different Meltano projects will be cumbersome by comparison.

Thanks @Will Da Silva (Arch) i didn't know about this. Let me try this out!