Hi. I've deployed Meltano using Kubernetes on Airf...
# best-practices
m
Hi. I've deployed Meltano using Kubernetes on Airflow. With enough dag runs you get an idea of the average runtime which includes the pod startup, Meltano run etc. which helps you decide the smallest safe schedule interval of the dag to prevent the next one starting before the previous one has finished. Interested in what others have implemented. To prevent this from happening I set a failsafe in the form of
max_active_tasks_per_dag=1
👀 1