Meltano & Airflow logs case: Is there a setting in Meltano which controls the meltano/airflow log files? For example: retain only last 7 or 14 days of logs. It currently stores big log files in those directories: .../.meltano/run/elt/... .../.meltano/run/airflow/logs For 30 days the log files are exceeding 50 GB and lack of space causes Meltano to shut down unexpectedly. Possible solution: Stumbled upon this solution: https://github.com/teamclairvoyant/airflow-maintenance-dags, but wanted to make ask if it is planned to integrate directly into Meltano, or is there a better way of cleaning up log files?

Thanks for bringing this up! Meltano doesn't currently do any logs cleanup, for its own logs or Airflow's, but that's definitely something we can look into. Can you please create a new issue? In the mean time you can set up your own cron job to automatically delete files with an old creation/modification date

Okay, will look into how to configure a cron job. I'll create a new issue.

@zlatan Thanks for creating the issue, I just responded: https://gitlab.com/meltano/meltano/-/issues/2465#note_459050578 Let me know if you'd like to look into implementing this yourself; I'll gladly provide some more guidance and code review 🙂

Great! Will check it

kind of an obscure question but if you run a meltano scheduler in a playbook that applies it to an ec2 instance, where would we find the logs?