Hello everyone, I am using Airflow as our Meltano ...
# troubleshooting
c
Hello everyone, I am using Airflow as our Meltano project orchestrator.... I am simply wondering when/how Meltano is creating the
airflow.cfg
file? If anyone could point that out for me that would be greatly appreciated!
p
Hey @connor_flynn - I dont know all the details about Airflow but if you check your
.meltano/run/airflow
directory you should see your log files and an airflow.cfg file. I'm not sure exactly how thats generated but I do know that you can set your configs using meltano.yml setting or using environment variables (I'm sure you've seen this already but https://docs.meltano.com/guide/production#airflow-orchestrator in the docs has an example). If you think what youre seeing in https://meltano.slack.com/archives/C01TCRBBJD7/p1653404799273549 is a bug then feel free to create an issue for someone to look into it, or if it turns out not to be a bug it could be worth while to create an issue to document the answers because its not obvious 😄
c
Thank you for the response @pat_nadolny! In the path
.meltano/run/airflow
I currently only have the following files: • airflow-scheduler.err • airflow-scheduler.log (this file is empty/blank) • airflow-scheduler.out • airflow.cfg (is not always present) I have set the Airflow configs for Executor and the system database using the environment variable option. When truing to set via the meltano.yml I get the following message (see screenshot) I am happy to open an issue... this has been an ongoing problem, that the Airflow community believes is Meltano related!
p
@connor_flynn yeah I'm seeing some weird behavior while I'm poking around. Can you open an bug issue for this?
c
Sure thing @pat_nadolny Getting that issue up now. I will share a link once posted!
@pat_nadolny I have got that issue opened now! Link to issue.
d
@connor_flynn Just commented on the issue 🙂
c
Thank you so much for the immediate response @douwe_maan !