📢 Office Hours starting in 1 hour. melty bouncy Come chat with the team, ask questions about Meltano, show off your latest creations, and discuss new features. You can find us in Zoom at the start of the hour! melty rainbow flame

Enjoyed the office hours yesterday, Thanks, had to drop off a little early. @visch you mentioned something that got me thinking. When talking about how to deploy, you mentioned you would prefer not to have to manage with docker if you didn't have to. I'm trying to upskill myself regarding docker, and saw that as a pretty clean way to get everything deployed. Just wondered what you envisaged as an alternative to a docker image?

Docker is the way to go in production. So I would stick with it. The alternative would be to directly install it on your os and run it just like most programs run on computers.

Ah that makes sense, you were just talking about the amount of effort involved to get from 'googling meltano' to seeing a first data transfer from tap to target running locally.

Well I do have a specific client in mind whos team is 100% windows and they don't have a lot of docker experience. Running in production would be easier for us if it wasn't in docker, but there's a number of things that don't quite work with large production deploys without docker (Some of them you'd be reimplementing docker for as they are deployment related) so 🤷 But yes stick with docker unless you have some reason not to!!

You basically described my situation up above so any general learnings greatly appreciated. I've found the meltano docs helpful building and running an image locally, but when it comes to container registries/deployment pipelines on Azure I'm pretty much just starting out.

1.

meltano run

works on windows now so you're in a great place already 2. Docker on Windows does work so use it! You can run it natively now it's kinda suprising that it works (moby project is what we use to avoid licensing) 3. docker prune is important to run once in a while (Folks who say a single host docker isn't good enough tend to point you to k8s or some other container orchestrator, but a single host for us works fine for our use case) 4. Building docker images isn't super quick so deployments can sometimes be slow, you can design a Dockerfile for "fast deploys" to speed up iteration times that don't reinstall everything with any

meltano.yml

file change 5. Your orchestrator is important, pick one you know. Make sure logs are handled and available easily somewhere 6. Iterate on your platform you design, focusing on speeding up dev cycles.

@visch appreciate this is an old thread, but: "Building docker images isn't super quick so deployments can sometimes be slow, you can design a Dockerfile for "fast deploys" to speed up iteration times that don't reinstall everything with any

meltano.yml

file change" I am ready to learn on this one now! 10 mins to deploy my image for a small change is starting to grate. Any tips gratefully received

New thread will give you a better chance, these get buried so fast for me