Is Postgres the only option for an external system database?

Hi Lukas! SQLite and MSSQL are also supported https://docs.meltano.com/concepts/project#system-database

Thank you! I'll look out for MySQL support.

Something else to be aware of is https://docs.meltano.com/concepts/state_backends. The major uses of the database are for run history and for storing incremental state between runs. Its pretty common for users to use a state backend like S3 and no external system db so they can keep their deployment lightweight. The downside is that you dont persist run history, although if you're using an external orchestrator like Airflow/Dagster or Meltano Cloud then run history is already managed there

I do believe I only need it to persist the state between runs. All our work is orchestrated using Argo Workflows.

@pat_nadolny when you say 'no external system db', do you mean 'leave it as the default sqlite and not worry about accessing it', or actually turn off the system db completely?

@Andy Carter I dont know of a way to turn it off but I just meant to leave it as the default sqlite. For example if you're running in an ephemeral container the job will write the job status to a local sqlite db on disk and the state to your S3 (or other) external state backend, then when the container spins down the sqlite db is deleted. When orchestrated with Airflow it doesnt matter because airflow already keeps track of that same job history, status, logs, etc. for you

That's what I thought, thanks. I guess you only thing you miss there is the 'locking' ability to avoid concurrent runs off the same tap and target combo.