Hello everyone I have a question Is it possible to trigger a Meltano ETL from a remote (meaning : another server) Airflow ?

Out of the box with airflow I think the answer is no. But you can setup airflow to ssh to a box and call commands. Maybe using ansible or just direct ssh. From a quick look at the Executor section Celery, and Dask could help too but if you're not suing them a "simple" ssh script might do the trick for you for now

Thank you very much @visch 🙏

Note I have never been an avid Airflow user but I've used a bunch of scheduling tools 🙂 and looked into Airflows docs more than once deeply

No but the this is that Meltano considers that scheduling happens directly in the same process which it rarely does

I don't follow you I guess, I'm missing something probably?

I mean when you install Meltano, it comes with an Airflow integration for orchestration. But what if (and that’s the most likely) you already have an Airflow running ?

https://meltano.com/docs/orchestration.html#using-an-existing-airflow-installation , it looks ike you can copy the dags from the orextrate/dags folder and put them on your existing Airflow server. By running

meltano add files airflow

If you wanted that would work to it looks like!

I’ll take a look at it

No problem let me know if that works, I'm curious!