Hey folks!
I’m trying to figure out how to re-use the meltano system db and attach additional metadata per run. I’m using airflow and use a followup task that generates metadata about a run.
More specifically, each meltano run has a run_id attached to it. This is unique for each run. I’d like to re-use this run_id in my own database so I can join with meltano systemdb and say “For this run_id, here’s the status, bookmarks, and my generated metadata”
But unfortunately I can’t find how to access the run_id. Any hints or alternate approaches?
Something else that could work is having some way of tagging each run, so I can re-use that tag to link to my generated metadata