Hi all, are there any best practices docs for deployment? Beyond the Deployment in Production page, I’m looking specifically for best practices around using Meltano to sync multiple schemas across multiple DBs on multiple instances of Postgres to Snowflake - should I break down Meltano projects by database? schema? tables? Should each project have its own container? Do I just make one container per DB with lots of inherited connectors and even more pipelines?

I’ve had success with a single docker image with every pipeline in it, but spin up a separate container for each pipeline execution, supplying context via env vars

@chris_kings-lynne that makes sense, since config costs nothing and you don’t have to keep track of whether you’re spinning up the right container for the right pipeline.

I use chamber to hydrate the env vars from Parameter Store

@chris_kings-lynne do you make config changes through the UI or CLI, or do you always rebuild the image if you need to change the config for a single pipeline?

i rebuild the image if any change to meltano.yml yep

thanks, super helpful. Did you try running multiple pipelines in a single container, perhaps a group in a shared domain or schema, and find drawbacks compared to a container:pipeline 1:1 ratio?

I can’t imagine any advantage in doign that over ephemeral containers

sorry, what I mean is doing what you’re doing but maybe one container bundles 3 pipeline executions instead of 1?