https://linen.dev logo
Join Slack
Powered by
I'm trying to upgrade meltano from 1.85.0 to 2.4.0...
# troubleshooting
q
I'm trying to upgrade meltano from 1.85.0 to 2.4.0, but I'm getting an error, 
Cannot upgrade the system database. It might be corrupted or was created before database migrations where introduced (v0.34.0)
. I've tried removing .meltano/meltano.db, but the error persists (meltano seems to create a new db and then fail at migrating it?). Has anyone run into this? Is there a way to just "reset" the system database and start from scratch? Honestly not understanding how deleting meltano.db is doing anything other than forcing meltano to re-init the db and skip migration(?)
Specifically, it's failing when trying to alter the job table,
Copy code
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) near "job_id": syntax error
[SQL: ALTER TABLE job RENAME job_id TO job_name]
(Background on this error at: <https://sqlalche.me/e/14/e3q8>)
Cannot upgrade the system database. It might be corrupted or was created before database migrations where introduced (v0.34.0)
e
@quinn_batten were you able to confirm or upgrade SQLite?
q
I'm using a EC2 box and it's looking like upgrading sqlite past 3.7.17 will reuqire me to build it myself
Which... I can do but is a bit of a pain. Sort of an AWS problem more than a meltano problem I guess
e
I’m curious: are you not relying on meltano’s system db to store state for your pipelines? If you are, you can simply point to an external database (postgres or mssql).
q
I'm using an external db for airflow, but meltano's system db for everything else
Gotcha, so maybe an easier fix would be to point it at a postgres db instead?
e
Yup, that would avoid the sqlite version problem
f
@quinn_batten just curious what linux flavor is your ec2 instance ? Amazon Linux 2 ?
q
Yep, Amazon Linux 2
Switching to a Postgres instance for meltano's system db does seem to have solved this. Thanks!
5 Views
Open in Slack
PreviousNext
Powered by