Hi guys, Just started to play around with the mapping feature and its been exciting so far. My goal is to generate dynamic configs for my mappings so that they can match the specific stream I have selected. I was wondering if it is possible to supply mapping config dynamically? Ideally, I would be able to do this with environment variables. Is there any way to get an environment variable to be read as an object (similar to the kind option for a setting?). For example, I want to be able to recreate the working config for

remove-properties

for

remove-properties2

. I also see that for this particular mapper it may be possible by supplying a path to a config file https://github.com/MeltanoLabs/meltano-map-transform/blob/b3a74f4b2865d0462ccbc70e523e0b6071eb7086/meltano_map_transform/mapper.py#L44-L59?

mappers:
  - name: meltano-map-transformer
    variant: meltano
    pip_url: git+<https://github.com/MeltanoLabs/meltano-map-transform.git>
    mappings:
    - name: remove-properties
      config: {"stream_maps": {"public-cdc_test": {"id": "id", "__else__": null}}}
    - name: remove-properties2
      config: ${MAPPER_REMOVE_PROPERTIES2_CONFIG}

Hi, @chris_evans! Can you share a bit more about the use case you have in mind?

For most of my plugins I am configuring them dynamically via environment variables. This allows me to run them in parallel against individual streams w/ separate state while also centralizing the config logic in my orchestrator.

Have you tried iterations of the

<PLUGIN_NAME>_<SETTING_NAME>

convention? Tbh, I don't know if we support this as of yet for mappers/mappings but if not, we definitely can+should log that as a feature proposal.

Yea not sure if I'm declaring this correctly, but this is the response I get

Error: Block filter-properties not found

.

mappers:
  - name: meltano-map-transformer
    variant: meltano
    pip_url: git+<https://github.com/MeltanoLabs/meltano-map-transform.git>

Trying the following env variables:

export MELTANO_MAP_TRANSFORMER_MAPPINGS='[{"name": "filter-properties", "config": {"stream_maps": {"public-cdc_test": {"id": "id", "__else__": null}}}}]'

export MELTANO_MAP_TRANSFORMER__MAPPINGS='[{"name": "filter-properties", "config": {"stream_maps": {"public-cdc_test": {"id": "id", "__else__": null}}}}]'