adam_tetz
06/12/2023, 2:40 PM{
"schedules": {
"job": [
{
"name": "daily-dora-load",
"interval": "@daily",
"cron_interval": "0 0 * * *",
"env": {},
"job": {
"name": "load-dora-data-to-s3",
"tasks": [
"tap-gitlab target-s3",
"tap-datadog target-s3"
]
}
}
],
"elt": []
}
}
When I start the airflow scheduler I see the following line in the scheduler log: [2023-06-12 16:06:58,373] {logging_mixin.py:115} INFO - [2023-06-12 16:06:58,373] {dagbag.py:508} INFO - Filling up the DagBag from /*removed*/meltano/orchestrate/airflow/dags/meltano_dag_generator.py
If I understand correct the dag generator should now register a dag. But the webinterface does not show a new dag. When I run meltano invoke airflow dags list
I see a dag_id that corresponds with the names of my schedule and job. However when I invoke this dag_id with meltano invoke airflow dags trigger dag_id
I receive an error: DagNotFound(f"Dag id {dag_id} not found in DagModel")
I haven’t changed any Airflow settings so everything is default.
Meltano runs in it’s own virtual environment. Python 3.10.12, Meltano version 2.19.1, Airflow 2.3.3
What am I missing?