Weird corner issue I just got stung by:

meltano.yml

- name: tap-mssql
    variant: buzzcutnorman
    pip_url: git+<https://github.com/BuzzCutNorman/tap-mssql.git>
    config:
      password: ${MY_PASSWORD}

but in .env

TAP_MSSQL_PASSWORD='wrong_password'
MY_PASSWORD='right_password'

Then the

TAP_MSSQL_PASSWORD

takes precedence over the specified config to use

MY_PASSWORD

when running the tap. Is this intended?

I believe this is actually expected, though I see why it’s confusing. I don’t know if https://docs.meltano.com/guide/configuration/#environment-variable-expansion does a good job of explaining all this as simply as it could. But I believe the tap would first look for

TAP_MSSQL_PASSWORD

and if it finds it just use that.

terminal env and .env

are where meltano first looks and then will pull from config.

I think what's a bit unexpected here is that it seems like

tap-mssql.config.password

in

meltano.yml

should take precedence (even if it requires expansion) over the env var. In reality the

<PLUGIN>_<SETTING>

has always taken precedence over

meltano.yml

: https://github.com/meltano/meltano/blob/34f713f11e70a294b0d1ac1cc26fbe4edb15737e/src/meltano/core/settings_store.py#L117-L125.

Thanks both for the explanation, not entirely clear if there is a bug here? Or just specific behaviour to be aware of? Obviously the right answer for my config is 'don't do that'!

> Thanks both for the explanation, not entirely clear if there is a bug here? Or just specific behaviour to be aware of? I think the latter really.

TAP_MSSQL_PASSWORD

takes precedence over

config: {...}

, otherwise, it wouldn't work as an override.

Thanks, I see it now. I think I misunderstood

CONFIG_OVERRIDE

as linking to

config:

in yaml, but that's

MELTANO_YML

which is further down the priority

That's correct, CONFIG_OVERRIDE is an internal implementation detail for things like

--log-level

that can override a setting value.