Do you have an example of python operator that cre...
# announcements
m
Do you have an example of python operator that create DAG's that invoke meltano pipelines?
d
@meir.shamay When you add the Airflow orchestrator to your project using
meltano add orchestrator airflow
, this default DAG generator should be added to your project automatically as well: https://gitlab.com/meltano/files-airflow/-/blob/master/bundle/orchestrate/dags/meltano.py
Of course, you can also use Meltano with an existing Airflow installation: https://meltano.com/docs/orchestration.html#using-an-existing-airflow-installation
m
So, basically I need to copy the generated DAG files from meltano machine to airflow machine?
d
@meir.shamay No, that shouldn't be necessary. You're already running Airflow it sounds like? How are you currently triggering workloads to run on different machines? Using the DockerOperator or KubernetesOperator? You could easily use either with a containerized Meltano project: https://meltano.com/docs/production.html#containerized-meltano-project-6
@meir.shamay Let me know if you'd like to jump on a call by the way, I'd be happy to give you some more context on how Meltano might fit in your existing setup!
m
👍
d
@meir.shamay Feel free to book a slot on my calendar using https://calendly.com/douwem/half-hour-meeting 🙂