Ian OLeary
09/16/2024, 2:21 PM- name: staging
config:
plugins:
loaders:
- name: target-snowflake
config:
database: STG
utilities:
- name: dbt-snowflake
config:
database: STG
For example i have this set for my stg env but when trying to materialize my models im getting "env var DBT_SNOWFLAKE_DATABASE required but not present" or something of the sort. I was under the impression you could set these parameters in the plugin config? Is that not true?Charles Feduke
09/16/2024, 2:44 PMDBT_SNOWFLAKE_DATABASE=whatever meltano…
and see if it functions that way/you get a new error messageIan OLeary
09/16/2024, 2:51 PMCharles Feduke
09/16/2024, 3:20 PMmeltano
command, you can just do:
DBT_SNOWFLAKE_DATABASE=whatever meltano invoke blah blah blah
(all on the same line)
and execute that and that will tell you whether its a bug with the plugin or misconfigurationIan OLeary
09/16/2024, 3:31 PMCharles Feduke
09/16/2024, 3:31 PMprintenv | grep DBT_SNOWFLAKE_DATABASE
does that return a value in your shell?Charles Feduke
09/16/2024, 3:33 PM.env
file or you’ve defined those environment variables elsewhere you’ll need to get them to your container… I would expect the dbt plugin for meltano to build these variables from configuration though (which is why you’re asking this question) but maybe it does not and you happen to have them set locallyIan OLeary
09/16/2024, 3:37 PMenv_file: .env
(only in development of course, in the cloud my env vars are set in a key vault and then loaded as container secrets and env vars in my container app). It looks like the meltano config isn't being loaded properly in my container but is being loaded fine in my local shell.
Running meltano config dbt-snowflake list
returns all the expected values for the non-sensitive configs. For instance:
`database [env: DBT_SNOWFLAKE_DATABASE] current value: 'STG' (from the active environment in meltano.yml
)`
`schema [env: DBT_SNOWFLAKE_SCHEMA] current value: 'DBO' (from meltano.yml
)`