If you have a docker image in prod running airflow, which is replaced by a CICD pipeline with each git push, how do you safely stop the container and replace it with the new one?

I’m still new to some of the internals of Airflow. However, if state and logging are tracked in a remote database (like Postgres), then the (in theory), you might be fine after reloading/rebooting the container from a fresh image. In terms of stopping the existing instance safely though, without interrupting jobs, do you have a time window where you know everything would be complete?

Yeah that’s sort of what I’m wondering, especially if an initial load which takes an hour is going on

I’m not sure if git integration would help this case, but it is something we have in our roadmap: I don’t know enough about your environment to know if pulling down the latest from git would help/hurt the processes you have already running. I think in general, probably it would be helpful as long as files aren’t modified while they’re being accessed. Do you have any thoughts on whether git pull would help this scenario? Re: Make project management in Meltano "git aware" (#2685) · Issues · meltano / Meltano · GitLab

I tend to like immutable containers 😕. I think I will instead run each ELT in a separate ephemeral container. can I still run one UI container that can monitor all the ephemeral ones?