Question about Containerization and the SQLLite DB. We have a scenario where we have one Meltano instance/container per tenant - we've moved the state files out of the container by using the

MELTANO_STATE_BACKEND_URI

and `MELTANO_STATE_BACKEND_AZURE_CONNECTION_STRING`ENV vars to be accessed via azure blob storage. What I failed to realize is that there's still a SQLLIte DB being utilized in the container https://docs.meltano.com/reference/settings?meltano-tabs=meltano+config#database_uri. I'm being told that when we tear down the container for any reason, or if it restarts, that SQLLiteDB is gone and we lose the metadata info, which is causing a bunch of issues. If it turns out that our best path forward is just to use a MSSQL / Postgres DB, we can't use the same DB to hold multiple states of different tenants at the same time right? It'd have to be one per tenant? Another Q is can we store the entirety of the

.meltano

structure in a blob storage within Azure via

$MELTANO_SYS_DIR_ROOT

? 🤔 https://docs.meltano.com/concepts/project/#meltano-directory

is causing a bunch of issues.

If you have an external state backend, I'm curious what issues are caused by losing the SQLite db?

A great question, I'm trying to ask that myself - my coworker is currently setting up the Azure/Infrastrucutre side so when I get more details I'll let you know asap

I see. Yeah, the dagster extension defaults

dagster_home

to `$MELTANO_PROJECT_ROOT/.meltano/dagster`: https://github.com/meltano/hub/blob/7790a2b38e5acde01570f8d2501433e52106c844/_data/meltano/utilities/dagster/quantile-development.yml#L43-L46 So you can probably point that to a different location and use a mount (or the equivalent Azure containers).

👍