Hi Folks, I am trying to integrate Meltano with existing Airflow setup which uses KubenetesExecutor. Ia it a good idea, Any suggestions? Thank you.

Hi @jayant_kumar, I think this is a good plan. We are also running meltano on kubernetes using the kubernetespodoperator from airflow. Meltano has a supporting utility to generate a docker file to run it within a container. And the meltano container can be spinned up on kubernetes like all others ;-) Regards, Bruno

+1 to this, we've been running quite a lot of Meltano jobs in kubernetes via Argo workflows as has been pretty good. One thing to note is pretty much any configuration for a tap or target can be overridden with environment variables, so I tend to keep the static config such as installed taps in a meltano.yml in a Dockerfile, and override run specific settings with environment variables. I also was pretty quick to use a postgres db for storing job state as I wanted to be able to introspect the state of the jobs

Thank you for the heads-up @bruno_hermans @ben_theunissen I am gonna try it then.