#7063 bug: Expansion of env vars in plugin command args misses `meltano.yml` env level Issue created by WillDaSilva Meltano Version 2.11.0 Python Version 3.10 Bug scope Configuration (settings parsing, validation, etc.) Operating System Linux Description version: 1 default_environment: dev project_id: f71334e3-2231-4661-baa5-ddc3aeb9f852 send_anonymous_usage_stats: false env: LEVEL: '2' STACK: ${STACK}-2 CHAIN2: ${CHAIN1}-2 plugins: utilities: - name: pyjokes namespace: pyjokes pip_url: pyjokes commands: joke: executable: pyjoke args: STACK=${STACK}, CHAIN1=${CHAIN1}, CHAIN2=${CHAIN2}, CHAIN3=${CHAIN3}, CHAIN4=${CHAIN4}, CHAIN5=${CHAIN5}, LEVEL=${LEVEL} env: LEVEL: '4' STACK: ${STACK}-4 CHAIN4: ${CHAIN3}-4 environments: - name: dev env: LEVEL: '3' STACK: ${STACK}-3 CHAIN3: ${CHAIN2}-3 config: plugins: utilities: - name: pyjokes env: LEVEL: '5' STACK: ${STACK}-5 CHAIN5: ${CHAIN4}-5 - name: staging - name: prod Using the above

meltano.yml

, we expect the following output from `meltano invoke pyjokes:joke`:

2022-12-08T20:19:15.638460Z [info     ] Environment 'dev' is active
usage: pyjoke [-h] [-c {neutral,chuck,all,twister}] [-l {en,de,es,gl,eu,it}]
pyjoke: error: unrecognized arguments: STACK=1-2-3-4-5, CHAIN1=1, CHAIN2=1-2, CHAIN3=1-2-3, CHAIN4=1-2-3-4, CHAIN5=1-2-3-4-5, LEVEL=5

Instead we get the following:

2022-12-08T20:19:15.638460Z [info     ] Environment 'dev' is active
usage: pyjoke [-h] [-c {neutral,chuck,all,twister}] [-l {en,de,es,gl,eu,it}]
pyjoke: error: unrecognized arguments: STACK=1-3-4-5, CHAIN1=1, CHAIN2=${CHAIN1}-2, CHAIN3=-2-3, CHAIN4=-2-3-4, CHAIN5=-2-3-4-5, LEVEL=5

Env var expansion in the top-level

env

field in

meltano.yml

normally works, as it is used within the env precedence integration test. I have confirmed that this is not a new issue as of Meltano 2.11.0 - it also affects Meltano 2.10.0, and Meltano 2.9.0. I did not test further. I am not sure exactly what fields are affected, aside from the

args

field of plugin commands. Running with

--no-environment

does not resolve the issue. We still see

CHAIN2=${CHAIN1}-2

in the output. As with most other env var related issues, I would recommend against prioritizing a fix for this until the Meltano manifest feature has been implemented: • #6876 Code No response meltano/meltano