Any tips on productionalizing Meltano? My project is Dockerized and am using CI/CD to build and push to a container registry. I'm currently just manually pulling the Docker image into the Ubuntu instance after the build, but would like to automate a little more, maybe creating a workflow for SSH to pull the image in. I haven't incorporated Airflow yet, but am going to need a scheduler soon, and thinking about the best way to run that in an Ubuntu instance.

Airflow would work, I personally love the simplicity of a crontab that runs

docker run image_name

, then for the repo implementing a deployment process that hops on that linux box and runs

docker pull image:latest

simple but really effective

Would you suggest mounting the

.meltano

directory in a volume?

I personally regen it in the container image, using uv makes this normally pretty quick (depending on plugin count) Yes state backend 😄 , you've got it!

Awesome, works great actually!

If you haven't pinned your taps and targets to the current versions you are using in

meltano.yml

, that can help to avoid unexpected changes if you are rebuilding often. https://docs.meltano.com/guide/plugin-management/#pinning-a-plugin-to-a-specific-version

@visch Are you using UV for Python dependencies, or Meltano plugin installation as well?

I believe there's something we can do here now that pylock.toml has landed