Hey all. Looking for some help with using stream maps with environment variables within the context of inherited plugins. Here are the docs I'm referencing for "declaring" the property I'd like to set a stream map up for in my meltano.yml config, so that I can then use an environment variable later on. The actual transformation I'm doing is very simple, it's just

str(fieldname)

. Here is my

settings

block in my config:

settings:
    - name: stream_maps.public-organization_connected_apps.metadata

metadata

happens to be the name of the column I'm trying to use in my stream map. Down stream, I set this up like so (executing from an Airflow environment):

env_vars["TAP_POSTGRES__POS_STREAM_MAPS_PUBLIC_ORGANIZATION_CONNECTED_APPS_METADATA"] = "str(metadata)"

And all of that works perfectly. But then I realized that the

settings

block is in my top level

tap-postgres

plugin config. This may cause an issue because I use several extractors that inherit from this top level config so that I can pull from multiple source databases. I'd like for the

settings

block to be pulled down into the inheriting extractor to avoid any column/property naming collisions. When I pull the

settings

block into the inheriting extractor, my tasks fail. The traceback is deep, but I can provide it. Ultimately, the error message is:

KeyError: 'stream_maps.public-organization_connected_apps.metadata'

Is this a bug? Am I doing something wrong? Thank you! --- • meltanolabs variant of

tap-postgres

v0.1.1 • meltanolabs variant of

target-snowflake

v0.15.1 • meltano version 3.6.0

Hi @taylor_van_anne!

I'd like for the

settings

block to be pulled down into the inheriting extractor to avoid any column/property naming collisions.

If you do this and then run

meltano config <inheriting extractor> list

, do you see expected env var for the setting?

Hey @Edgar Ramírez (Arch.dev)! I don't. I just see the name of my stream map printed out in red. And if I do this with debug log level, then I see the same traceback as my failed task.