Hi Team, I am looking for to trigger Meltano job from external airflow (mwaa). can you please suggest the way ? api or ssh or bash any possible ways?
a
atharh
04/22/2022, 2:49 PM
Hi @rahul_patel. You can use Airflow’s BashOperator to execute commands in a Bash shell. Does this help?
r
rahul_patel
04/22/2022, 4:22 PM
Airflow and meltano are in different environment. Airflow is aws managed and meltano is EKS pods.
p
pat_nadolny
04/25/2022, 1:32 PM
hey @rahul_patel - could you use the KubernetesPodOperator to spin up the Meltano container with your command? Thats what we do in our Meltano instance using a custom implementation to just prep some inputs first before calling the standard KubernetesPodOperator