Meltano

Is there a way to read the running logs of a Meltano container that's running airflow? We have our Meltano containerized and running our pipelines within AWS, but we don't know where to find logs for what each pipeline is doing.

<@U06CQMUDR09> Have you considered spinning up a container running `meltano invoke airflow webserver` or `meltano ui` alongside the container running `meltano invoke airflow scheduler` ?

Either should let you browse the current pipelines and view their current/latest logs, assuming you've set up Airflow (or Meltano itself) to store logs somewhere where multiple containers can access it

If you're using Meltano UI, that means <https://meltano.com/docs/production.html#storing-logs>

If you're using Airflow webserver, that means something like <https://airflow.readthedocs.io/en/stable/howto/write-logs.html#writing-logs-to-amazon-s3>

So we've got the container set up to immediately run `meltano invoke airflow scheduler`. We should then be able to go into `meltano ui` within the `pipelines` tab and look at each of the logs?

Yep, as long as both containers have access to the same system database and log directory, as explained in the production docs :slightly_smiling_face: