How does Meltano track a tap's Singer state? Does the state get written/saved in some sort of internal Meltano file, or is it saved via your orchestrator's metadata?

@michael_cooper https://meltano.com/docs/integration.html#incremental-replication-state

Meltano stores this pipeline state in its system database, identified by the

meltano elt

run's Job ID.

https://meltano.com/docs/project.html#system-database

job

table: One row for each

meltano elt

pipeline run, holding started/ended timestamps and incremental replication state.

So if the data is stored in the system database, does that mean you lose the state if your production container goes down or gets updated?

Only if the system database is stored inside the container, not if you follow this recommendation: https://meltano.com/docs/production.html#storing-metadata

Thus, it is is strongly recommended that you use a PostgreSQL system database in production instead. You can configure Meltano to use it using the `database_uri` setting.

So if using Airflow and has been set up with an external database, you also need to set the

database_uri

to also use an external database?

Correct, Meltano and Airflow use their own metadata databases, and they should both be external in production