hi folks, i'm looking for some advice on how to parse environment variables: context: • we store connection secrets in aws secrets manager, where the secret string is like

http://{username}:{password}@host/path?param1=value1&param2=value2...

• our devops team will put that string into the meltano container when it runs, so in the particular example, we'd end up with an environment variable

MARKETO_CREDENTIALS

with value

http://{client_id}:{client_secret}@xxx-xxx-xxx.mktorest.com/rest?MKTO_IDENTITY={mkto_identity_url}&MKTO_MUNCHKIN_ID={mkto_munchkin_id}

• is there a way to specify in the

marketo.yml

config (or elsewhere) to extract the credentials out of this pattern?

If they could split out the ENV variables it'd make your life easier! I'd assume they could 🤷 https://meltano.com/docs/configuration.html#configuring-settings , can reference pretty much whatever environment variables you'd like. Transforming the environment variables is a different question though I'm not sure if it can do regex or something on the variable to pull out the data you want

yeah, i was thinking about just putting in a short python script to run which would split up & set environment and the container run command can be something like

. ./set-env-vars && meltano tap-name target-name

You might be able to use https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html I think https://meltano.com/docs/configuration.html#expansion-in-setting-values is using standard expansion syntax so you could probably do some regex magic here. If this works please share 😄

I use and recommend Chamber