Does anyone have a working example of their dbt co...
# troubleshooting
i
Does anyone have a working example of their dbt config in their meltano.yml? I keep trying to set my db/warehouse/role/schema config in my meltano.yml but it keeps saying "env var not found" for DBT_SNOWFLAKE_DATABASE or role or whatever it is but I have all of those set in my meltano.yml
Copy code
- name: staging
  config:
    plugins:
      loaders:
      - name: target-snowflake
        config:
          database: STG
      utilities:
      - name: dbt-snowflake
        config:
          database: STG
For example i have this set for my stg env but when trying to materialize my models im getting "env var DBT_SNOWFLAKE_DATABASE required but not present" or something of the sort. I was under the impression you could set these parameters in the plugin config? Is that not true?
c
you could try running
DBT_SNOWFLAKE_DATABASE=whatever meltano…
and see if it functions that way/you get a new error message
i
What do you mean? Like set the env var in config? Or actually declare it in my .env?
c
well in your shell when you run the
meltano
command, you can just do:
Copy code
DBT_SNOWFLAKE_DATABASE=whatever meltano invoke blah blah blah
(all on the same line) and execute that and that will tell you whether its a bug with the plugin or misconfiguration
i
Well the weird thing is it's invoking fine from my shell. The issues only happens once I try materializing in my container
c
printenv | grep DBT_SNOWFLAKE_DATABASE
does that return a value in your shell?
if you have a
.env
file or you’ve defined those environment variables elsewhere you’ll need to get them to your container… I would expect the dbt plugin for meltano to build these variables from configuration though (which is why you’re asking this question) but maybe it does not and you happen to have them set locally
i
I'm setting sensitives locally in my .env and then building that env into my container with dockercompose with
env_file: .env
(only in development of course, in the cloud my env vars are set in a key vault and then loaded as container secrets and env vars in my container app). It looks like the meltano config isn't being loaded properly in my container but is being loaded fine in my local shell. Running
meltano config dbt-snowflake list
returns all the expected values for the non-sensitive configs. For instance: `database [env: DBT_SNOWFLAKE_DATABASE] current value: 'STG' (from the active environment in
meltano.yml
)` `schema [env: DBT_SNOWFLAKE_SCHEMA] current value: 'DBO' (from
meltano.yml
)`