I recently upgraded the Meltano version we are using to the Meltano #troubleshooting

I recently upgraded the Meltano version we are usi...

ben_theunissen

02/26/2024, 4:32 PM

I recently upgraded the Meltano version we are using to the latest CLI, and I've started to see a flaky error appear in my integration tests. On each test run we create a new postgresql db container for Meltano, run the Meltano

upgrade

command to update the schema, and then run our tests. I am finding occasionally that the following error is thrown from Meltano when running the

meltano el

command due to migrations being applied automatically and then failing. I am a bit confused as the migrations should already be applied to the db at this stage, and I haven't seen this behaviour before where Meltano is attempting to run the migrations anyway, anyone else seen this issue?

✅ 1

ben_theunissen

02/26/2024, 4:33 PM

full logs

ben_theunissen

02/26/2024, 4:35 PM

The strange thing is that the other runs of the tests in the same workflow are passing fine, it seems like occasionally alembic gets stuck

ben_theunissen

02/26/2024, 6:56 PM

After some more investigation seeming like this only happens when the tests start in parallel

👀 1

ben_theunissen

02/26/2024, 7:46 PM

Found the issue, I had setup code to migrate the MeltanoDB before any tests were ran, but that step began to silently migrate against SQLite instead of the PostgresDB due to a misconfiguration, which meant the DB migrations were being applied to the DB in parallel by the tests. Fixed by properly migrating the DB before any tests are ran.

👍 1

Open in Slack

Previous Next