hi, we have multiple databases using the same Tap and Target, what is the best way to define them? I see two options 1) plugin inheritance in meltano (will be tricky for lower environment, because meltano.yml is hardcoded) , 2) airflow DAGs with ENV override

@jingkun_zhuang Both options you mention are fully supported, but defining multiple (inheriting) plugins on the Meltano side is more explicit and scalable, since you can manage the different configurations separately and only use environment variables to inject sensitive values (like creds). Why do you think this approach will be tricky?

it is in meltano.yml, which will be same across all envs. Say I have 4000 prod databases to run, but I only have 2 test db in test environment

Ah, makes sense. In that case dynamically injecting different configurations through the environment would be the more scalable solution, if the tap is otherwise configured and used the same way each time

or is it possible to use different meltano.yml for different envs

No, the idea is that you'd use one project file in all environments, and use .env or the actual environment to switch out environment-specific configuration