Hi guys, I'm trying out meltano but there is something I don't fully understand - I want to run it with a few other services in docker compose. What commands should I be running? Should I just keep the container alive and manually

meltano init

in there? I prefer to use vs code dev containers so I would be developing inside the container. I think in this case just taking a Python image and pip installing Meltano OR overriding the meltano entrypoint of the image makes sense. What do you guys think?

I run meltano in docker as well (with Prefect), meltano init isn’t actually required. Once you have a meltano file, “meltano install” works. An example: https://github.com/radbrt/prefect_flows/tree/main/projects/jottings/meltano_flow

Good one. Thanks for linking your repo. Right now I solved it by just taking a python image, installing python in it and using

tty: true

to keep a terminal running.

@chidi_nweke yeah +1 to what Henning said. You only need to run init when youre creating a project for the first time. The docs suggestion to run docker for the CLI is a short lived container, it would spin up to execute your CLI command like

docker run -v $(pwd):/projects -w /projects meltano/meltano init my-meltano-project

then spin down, leaving the altered artifacts like meltano.yml on your local machine because its mounted as a volume. Meltano doesnt have any long running services so its not required to be always up but your idea to develop inside a meltano container seems reasonable. I think its pretty common for people to develop locally without docker (i.e. with meltano installed in a venv or using pipx) then package their meltnao project artifacts use docker, thats what I do too https://github.com/meltano/squared/blob/main/deploy/meltano/Dockerfile.meltano

Thanks for the very detailed answer @pat_nadolny half of my issues were indeed just not fully understanding docker and the docker image of meltano. Your clarifications help a lot in that respect. Currently I will try the in-container development strategy and see what that gives however a short lived container or your approach of making a container afterwards makes perfect sense.