Hi there. I'm looking to use Meltano to simplify the way we extract data from our Prometheus instances. We've been accomplishing this via Python in Prefect flows but that's becoming tedious and requires a lot of boilerplate code. I've made a very simple Prometheus-Tap, which executes instant or range queries and works just great but I'm running into issues with stream naming collisions. I have a set of plugins that inherit from a base prometheus-tap plugin, similar to this example. I want to specify/override the stream name per plugin so that when I use a BQ loader the data flows into the appropriately named load tables. I found the mapping plugin that allows me to alias streams, and I can set a static name in the

__alias__

property of the mapper plugin, but it doesn't seem to let me define the alias as an environment variable like other plugin config blocks allow. Ideally I can set an environment variable or use some other setting to allow me to specify an alias attribute in that mapper plugin for each extractor plugin. Looking at the code, an attribute (

parse_env_config

) seems to be what I want to configure but I can't figure out how to override the default. Am I missing a config setting or doing something incorrectly? Would I need a unique mapper for each plugin in this case?

Hi @bob_samuels ! Here’s a neat potential solution for you (that we should probably document): 1. Define the stream map setting using the dot separator:

plugins:
  extractors:
  - name: tap-pocket
    variant: edgarrmondragon
    pip_url: git+<https://github.com/edgarrmondragon/tap-pocket.git>
    settings:
    - name: stream_maps.items.__alias__
      value: items

2. By default the alias is the same, but you could leave out

value

Now you can override the value using the

TAP_POCKET_STREAM_MAPS_ITEMS___ALIAS__

environment variable:

TAP_POCKET_STREAM_MAPS_ITEMS___ALIAS__='items_v2' meltano invoke tap-pocket

Thanks @edgar_ramirez_mondragon! That worked for me.