Hi, I keep running into an issue with the system database on my meltano runs. I consistent receive the following error:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked

and I'm not sure what is causing the issue. It is especially confusing to me as I have configured a different system database through the `meltano.yml`:

version: 1
default_environment: dev
include_paths:
  - "*.meltano.yml"
state_backend:
  uri: <azure://meltano-state>
database_uri: $MELTANO_DATABASE_URI

With the

MELTANO_DATABASE_URI

, I'm pointing to a mssql database. I do see that the runs are logged here in the runs table, so the connection seems to work fine. Does anyone have an idea what might be causing this issue and suggestions on how to resolve it?

do you have log output?

I'd try deleting the sqlite db in

.meltano/meltano.db

. Should be ok since you're not using it.

@Charles Feduke you just mean the error log output right? That would be this:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: INSERT INTO event_logs (run_id, event, dagster_event_type, timestamp, step_key, asset_key, partition) VALUES (?, ?, ?, ?, ?, ?, ?)]
[parameters: ('e11126b8-83f0-40db-a57a-3ac900910573', '{"__class__": "EventLogEntry", "dagster_event": null, "error_info": null, "level": 20, "message": "", "pipeline_name": "contact_customers", "run_id": ... (192 characters truncated) ... ngCursor: 65 millis spent executing: COMMIT cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake"}', None, '2024-08-17 07:02:43.046607', 'tap_online_afspraken_rosmalen', None, None)]
(Background on this error at: <https://sqlalche.me/e/14/e3q8>)

  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/dagster/_core/instance/__init__.py", line 252, in emit
    self._instance.handle_new_event(
  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/dagster/_core/instance/__init__.py", line 2408, in handle_new_event
    self._event_storage.store_event(events[0])
  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/dagster/_core/storage/event_log/sqlite/sqlite_event_log.py", line 240, in store_event
    conn.execute(insert_event_statement)
  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1385, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement
    ret = self._execute_context(
  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context
    self._handle_dbapi_exception(
  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2134, in _handle_dbapi_exception
    util.raise_(
  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
    self.dialect.do_execute(
  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)

The above exception was caused by the following exception:
sqlite3.OperationalError: database is locked

  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
    self.dialect.do_execute(
  File "/root/.cache/pypoetry/virtualenvs/dagster-deployment-yBSkMgww-py3.9/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)