I'm trying to think through devops and using meltano in production. Specifically, config values. The .env file and meltano.db are in .gitignore. Are there any recommendations or practices around deploying and around managing production config values?

@adam_roderick Right now, I would suggest using

config

objects on your plugins' definitions in

meltano.yml

for non-sensitive values, which I intend to make more accessible through

meltano config

soon: https://gitlab.com/meltano/meltano/-/issues/2013. For sensitive values, environment variables are your best bet since many systems exist to manage them in production environments, some of which also support hiding the most sensitive ones in vaults. I would love it if Meltano would develop into a more scalable, more mature direction on this front. The 2 paragraphs I just wrote above could be the start of a "Production" section in the docs, and I think we should discuss more forward-thinking solutions in an issue. Since you appear to have some thoughts on the topic, I would suggest creating an issue to get the conversation started, and I will update the docs with what I wrote above: https://gitlab.com/meltano/meltano/-/issues/2038

I imagine a good Docker workflow for meltano would allow one to set config for each component via environment variables, picked up at runtime by meltano (using something like

<https://github.com/segmentio/chamber>

in a production setting). An example of a local environment using docker compose (meltano + meltano db + some source + some target) would be incredibly helpful.

An example of a local environment using docker compose (meltano + meltano db + some source + some target)

would be incredibly helpful.

I agree this would be useful, and I created a related issue at https://gitlab.com/meltano/meltano/-/issues/1970, although my thinking there was that this would be primarily useful for people wanting to use Meltano UI. In any case, what you describe should already be possible in principle—perhaps you would like to have a crack at setting it up and I can help out if you run into any issues?