Hi! What is the idiomatic way to have a look at the logs? In a similar vain: Is there a way to monitor scheduled ETL? I installed the Airflow Plugin and Airflow shows runs either as 'failed' or 'running'. My guess is that airflow does not register success and 'running' for a long time indicates success. 😉

Meltano stores its own logs at

.meltano/logs/elt

, and Airflow will store its logs wherever you configure it to. https://airflow.apache.org/docs/stable/howto/write-logs.html Airflow will register success when the operator it is using (by default BashOperator generated by the dag generator) returns a successful exit code. There is no bespoke coupling here between Airflow and Meltano beyond Airflow simply calling the

meltano elt

commands, and waiting for the result in the same way it would with any other task

Niall is exactly right; thanks for taking this question for me! 😄 As far as viewing logs is concerned, there's an open issue to add a

meltano logs <job ID>

command, since logs per pipeline are currently only readily available in the UI: https://gitlab.com/meltano/meltano/-/issues/2228 Of course, you can also view the logs manually at

.meltano/logs/elt/<job_id>/<run_id>/elt.log

, as documented in https://meltano.com/docs/project.html#meltano-directory.