Really having trouble running through one last hurdle on a k8s deployment of meltano, this error is happening during pod initialization. I am using the meltano/meltano base image and followed the setup guide pretty closely. Using postgres for a database for both meltano and airflow. Airflow is invoked via meltano invoke. I dont understand why alembic needs to do the database migration if its a fresh database in the cluster

alembic.util.exc.CommandError: Can't locate revision identified by '13e8639c6d2b'
...
meltano.cli.utils.CliError: Airflow metadata database could not be initialized: `airflow initdb` failed
Airflow metadata database could not be initialized: `airflow initdb` failed

I am getting this constantly and the Pod is crashing and backing off. Anyone seen this before or could help? Very appreciated.

It appears that meltano and airflow are trying to use the same table to manage alembic migrations which won’t work. I would expect that you would want to create two different databases on your postgres server, one for meltano and one for airflow. Or in some other way tell meltano and airflow to use different tables or schemas for alembic.

yeah you were right, thats why you shouldnt stare at yaml too late at night 😛 thanks!

Out of curiosity how are you running the K8s deployment? Did you run through the Helm Chart?

Yeah thats exactly how I am doing it 😉

Coolio

I actually made my own helm chart before digging into the Meltano one. I am using mine which has postgres as a dependency which meltano and airflow both use, airflow uses the built in meltano airflow venv so I can leverage meltano environments. Harness next gen CD + git experience (harness config as code) lets me have the entire data pipeline infrastructure deploy on git push/merges.

Interesting! Also Helm can go direct to the cluster just need that kube config also, what ever is easier.