Hey there! I’m trying out the Airflow utility and ...
# troubleshooting
a
Hey there! I’m trying out the Airflow utility and I can’t get meltano to generate a DAG and show it in the web interface. I followed all the steps in https://hub.meltano.com/utilities/airflow/#getting-started I created two jobs that I added to a schedule. The output of `meltano schedule list --format=json`:
Copy code
{
  "schedules": {
    "job": [
      {
        "name": "daily-dora-load",
        "interval": "@daily",
        "cron_interval": "0 0 * * *",
        "env": {},
        "job": {
          "name": "load-dora-data-to-s3",
          "tasks": [
            "tap-gitlab target-s3",
            "tap-datadog target-s3"
          ]
        }
      }
    ],
    "elt": []
  }
}
When I start the airflow scheduler I see the following line in the scheduler log:
[2023-06-12 16:06:58,373] {logging_mixin.py:115} INFO - [2023-06-12 16:06:58,373] {dagbag.py:508} INFO - Filling up the DagBag from /*removed*/meltano/orchestrate/airflow/dags/meltano_dag_generator.py
If I understand correct the dag generator should now register a dag. But the webinterface does not show a new dag. When I run
meltano invoke airflow dags list
I see a dag_id that corresponds with the names of my schedule and job. However when I invoke this dag_id with
meltano invoke airflow dags trigger dag_id
I receive an error:
DagNotFound(f"Dag id {dag_id} not found in DagModel")
I haven’t changed any Airflow settings so everything is default. Meltano runs in it’s own virtual environment. Python 3.10.12, Meltano version 2.19.1, Airflow 2.3.3 What am I missing?