There's a handy connection with the extractor `loa...
# plugins-general
s
There's a handy connection with the extractor
load_schema
and the loader
schema
config for
target-snowflake
. I'm using the
pipelinewise
variant, and it looks like it adds the following
default_target_schema: $MELTANO_EXTRACT__LOAD_SCHEMA
. However, I noticed that if I change the loader's name (e.g. to
target-snowflake-prod
, that "auto" connection breaks, and i have to manually add the config line. It raises a question for me -- where is that "auto" functionality being set? It's documented in Meltano Hub (https://hub.meltano.com/loaders/snowflake--transferwise.html), but I didn't change any executables, only the namespace and name. When Meltano executes, does it check the hub for additional configs?
also, i just noticed the #C01TCRBBJD7 channel and will try to direct these questions there rather than this very full channel 🙂
d
If you’ve changed the
namespace
on
target-snowflake-prod
, it won’t pick up the defaults from https://gitlab.com/meltano/meltano/blob/master/src/meltano/core/bundle/discovery.yml#L1802 anymore, including the default value for `default_target_schema`: ``MELTANO_EXTRACT__LOAD_SCHEMA``
So I’d suggest either not overriding
namespace
, so that it uses the defaults, or explicitly setting
default_target_schema: $MELTANO_EXTRACT__LOAD_SCHEMA
in
meltano.yml
s
does
namespace
drive the environment variable settings (i.e.
TARGET_SNOWFLAKE_PROD_USER
)? I was planning to have
target-snowflake-prod
and
target-snowflake-dev
just to make it easier to work with in orchestration / clearer for the team on what they should / should not use
and that
discovery.yml
file is a big ooooooh moment for me. i'll look into this a bit
t
I was planning to have 
target-snowflake-prod
 and 
target-snowflake-dev
 just to make it easier to work with in orchestration / clearer for the team on what they should / should not use
This gets into a topic we discussed at office hours about building a native understanding of environment into Meltano. @aaronsteers were you still planning on making an issue around that?
d
@stephen_bailey the name drives the env var, but you shouldn’t need to override the namespace if you use inheritance from the base target-snowflake
a
@taylor - Thanks for the friendly ping. I've just logged the "environments" feature proposal issue here: https://gitlab.com/meltano/meltano/-/issues/2869 This proposal would add additional capabilities regarding re-use of environment variables and config values across targets, dbt, and potentially other plugins in the future like validators and analyzers.