I am setting up a tap-postgres that will load data...
# plugins-general
n
I am setting up a tap-postgres that will load data from tables created as part of my inline dbt project. I am trying to figure out how to use environment variables for DBT schema environment variables in the tap
select
configuration. I have the following environment variables: DBT_POSTGRES_SCHEMA - this is the environment schema prefix for dbt (dev, prod, etc) DBT_TARGET_SCHEMA - I see this in my profiles.yml for dbt and see that it corresponds to what I set in the schema config for a given model. So my full schema + table select for a given environment is in theory: ${DBT_POSTGRES_SCHEMA}${DBT_TARGET_SCHEMA}-some_table However this doesn't work and I am not finding an example of how to specify this the right way. Any ideas?
s
Hey @nicholas_van_kuren, this is not yet going to work (https://github.com/meltano/meltano/issues/6770), as Meltano doesn't yet support variable interpolation in the select blocks. So the solution would be to have separate copies of your plugin, use inherit and overwrite the select blocks for you different environments. (@ken_payne mind checking this as well?)
k
@Sven Balnojan you are right in saying this isn't supported right now. Depending on the number of combinations of values for those two env vars, plugin inheritance may not be needed. @nicholas_van_kuren if
DBT_POSTGRES_SCHEMA
and
DBT_TARGET_SCHEMA
are static per environment, you can use the existing selection statements along with Meltano Environments. This works best if there is a table or schema naming pattern with a prefix or suffix that you can easily filter on. e.g.