Trying to put DBT configurations in my .env

transformers:
  - name: dbt
    pip_url: dbt-postgres==1.0.0
    settings:
    - name: password
      kind: password
    config:
      target: postgres
      host: localhost
      port: 5432
      user: postgres
      dbname: postgres

Then I run

meltano config dbt set password abcde

and meltano puts the password in meltano.yml, hmm. Maybe my expectations are wrong here (I have the same password in 3 places on this project, tap-postgres, target-postgres, and dbt. )

If you have

password

set to any value inside the plugin

config

, it will take precedence and meltano will overwrite it there iirc

message has been deleted

Thanks for recording the quick video. Based on the assumption that

kind: password

is set, I don't think the value should be landing in meltano.yml.

I've never tried doing this with transforms before. All of the time with extractors / loaders. Somethings different here, I can throw an issue in if you'd like sounds like I passed the smoke test

The behavior should be automatic - to store in either

dotenv

or

db

. Here's the syntax for explicitly setting a storage location, as a workaround at least:

# Set, unset, or reset in a specific location
meltano config <plugin> set --store=meltano_yml <name> <value> # set in `meltano.yml`
meltano config <plugin> unset --store=dotenv <name> # unset in `.env`
meltano config <plugin> reset --store=db # reset in system database

@visch what happens if you add a

namespace

to your dbt plugin definition?

Also

PS C:\code\bamboohr2ad> meltano config dbt set --store=dotenv password cb7cd89dc59e4da7b249e4f8f7c929d6
Transformer 'dbt' setting '['password']' could not be set in `.env`:

No workie

So it seems • A plugin with an explicit namespace in

meltano.yml

is considered custom by meltano • A plugin without one means it was discovered. So its settings are discovered too • What you put in

settings

for a discovered plugin is completely ignored by meltano

got it that explains

PS C:\code\bamboohr2ad> meltano config dbt list
project_dir [env: DBT_PROJECT_DIR] current value: 'C:\\code\\bamboohr2ad/transform' (default: '$MELTANO_PROJECT_ROOT/transform')
profiles_dir [env: DBT_PROFILES_DIR] current value: 'C:\\code\\bamboohr2ad/transform/profile' (default: '$MELTANO_PROJECT_ROOT/transform/profile')
target [env: DBT_TARGET] (default: '$MELTANO_LOAD__DIALECT') current value: 'postgres' (from `meltano.yml`)
source_schema [env: DBT_SOURCE_SCHEMA] current value: None (default: '$MELTANO_LOAD__TARGET_SCHEMA')
target_schema [env: DBT_TARGET_SCHEMA] current value: 'analytics' (default)
models [env: DBT_MODELS] current value: '  my_meltano_project' (default: '$MELTANO_TRANSFORM__PACKAGE_NAME $MELTANO_EXTRACTOR_NAMESPACE my_meltano_project')

Custom, possibly unsupported by the plugin:
host [env: DBT_HOST] current value: 'localhost' (from `meltano.yml`)
port [env: DBT_PORT] current value: 5432 (from `meltano.yml`)
user [env: DBT_USER] current value: 'postgres' (from `meltano.yml`)
dbname [env: DBT_DBNAME] current value: 'postgres' (from `meltano.yml`)

Custom extras, plugin-specific options handled by Meltano:
_settings [env: DBT__SETTINGS] current value: [{'name': 'password', 'kind': 'password'}] (from `meltano.yml`)

exactly 😄

@edgar_ramirez_mondragon - Wow, nice sleuthing! chef kiss You are probably already on this, but do you mind adding this summary and proposed fixes/recommendations to Derek's issue?

It really could just be me, and I should just accept that I need to learn the discovery mechanisms. I've been ignoring them completely

🙂 The presence of namespace silently changing the behavior seems problematic to me. I'm glad you logged.