#6978 bug: env inheritance Issue created by visch Meltano Version 2.8.0 Python Version 3.8 Bug scope CLI (options, error messages, logging, etc.) Operating System Ubuntu Description Top level environment variables don't seem to expand properly. There may be multiple layered issues here like the fact that I can dump the configuration and we don't get a failure with the Feature Flag set (not a huge deal but it was another layer of confusion) meltano.yml

version: 1
default_environment: dev
project_id: fa511b29-9a3f-4498-8e49-f9d7d79740dc
environments:
- name: dev
  config:
    plugins:
      loaders:
      - name: target-jsonl
        config:
          dev_env: ${TOP_LEVEL_ENV}
- name: staging
- name: prod
plugins:
  loaders:
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl
env:
  MELTANO_FF_STRICT_ENV_VAR_MODE: "true"
  TOP_LEVEL_ENV: ${MELTY_VALUE}

Run 1. meltano install 2. meltano --environment=dev invoke --dump=config target-jsonl (Should probably fail but ok looks alright) 3. meltano --environment=dev invoke target-jsonl (Fails, cool the feature flag works) 4. export MELTY_VALUE="MELTY" 5. meltano --environment=dev invoke target-jsonl (Doesn't fail as the value is set) 6. meltano --environment=dev invoke --dump=config target-jsonl (Show's that even though the feature flag see the value set there's still an issue) Run in action (terminal) ("${MELTY_VALUE}") at the end should match what we have in our environment.

visch@visch-ubuntu:~/git/meltano-projects/env_inheritance$ meltano --environment=dev invoke --dump=config target-jsonl
2022-11-11T21:08:22.195718Z [info     ] Environment 'dev' is active
{
  "destination_path": "output",
  "do_timestamp_file": false,
  "dev_env": "${MELTY_VALUE}"
}
visch@visch-ubuntu:~/git/meltano-projects/env_inheritance$ meltano --environment=dev invoke target-jsonl
2022-11-11T21:08:28.806785Z [info     ] Environment 'dev' is active
Environment variable 'MELTY_VALUE' referenced but not set. Make sure the environment variable is set.
visch@visch-ubuntu:~/git/meltano-projects/env_inheritance$ export MELTY_VALUE="MELTY"
visch@visch-ubuntu:~/git/meltano-projects/env_inheritance$ meltano --environment=dev invoke target-jsonl
2022-11-11T21:09:03.569423Z [info     ] Environment 'dev' is active
^CTraceback (most recent call last):
  File "/home/visch/git/meltano-projects/env_inheritance/.meltano/loaders/target-jsonl/venv/bin/target-jsonl", line 8, in <module>
    sys.exit(main())
  File "/home/visch/git/meltano-projects/env_inheritance/.meltano/loaders/target-jsonl/venv/lib/python3.8/site-packages/target_jsonl.py", line 92, in main
    state = persist_messages(
  File "/home/visch/git/meltano-projects/env_inheritance/.meltano/loaders/target-jsonl/venv/lib/python3.8/site-packages/target_jsonl.py", line 40, in persist_messages
    for message in messages:
KeyboardInterrupt

Aborted!
2022-11-11T21:09:12.204652Z [warning  ] Loop <_UnixSelectorEventLoop running=False closed=True debug=False> that handles pid 13734 is closed
visch@visch-ubuntu:~/git/meltano-projects/env_inheritance$ meltano --environment=dev invoke --dump=config target-jsonl
2022-11-11T21:09:21.492625Z [info     ] Environment 'dev' is active
{
  "destination_path": "output",
  "do_timestamp_file": false,
  "dev_env": "${MELTY_VALUE}"
}

(Had an example that I thought worked but I suddenly couldn't replicate it today hmm, removed) meltano/meltano