mert_bakir
05/30/2023, 11:57 AMplugins:
transformers:
- name: dbt-postgres
variant: dbt-labs
pip_url: dbt-core~=1.5.0 dbt-postgres~=1.5.0
commands:
run_dynamic_env:
args: run --target ${DBT_TARGET_ENV}
jobs:
- name: run-dbt--full
tasks:
- dbt-postgres:clean dbt-postgres:run_dynamic_env
schedules:
- name: schedule01
interval: 00 23 * * *
job: run-dbt--full
env:
DBT_TARGET_ENV: target_name
I want to create the same schedule for different targets, so I thought I can leverage schedule level env variables but it doesn't work:
Run invocation could not be completed as block failed: Cannot start plugin: Command 'run_dynamic_env' referenced unset environment variable '${DBT_TARGET_ENV}' in an argument. Set the environment variable or update the command definition.
The variable DBT_TARGET_ENV
is not defined anywhere else but only at the schedule level.
When I define it on root plugin level it does not get overridden by the schedule level definition. Same applies to .env file. Meltano picks up the variable from root plugin level and .env file but not from schedule. meltano, version 2.18.0mert_bakir
05/30/2023, 12:00 PMmert_bakir
06/01/2023, 9:25 AMpat_nadolny
06/01/2023, 1:42 PMmert_bakir
06/03/2023, 1:12 PM