Matt Menzenski
03/06/2024, 12:00 PMMELTANO_DATABASE_URI
to a SQLAlchemy Postgres URI), meltano writes to a number of Postgres tables - not just state
, but also runs
(and others, but runs
is valuable).
When running Meltano with an S3 state backend (setting MELTANO_STATE_BACKEND_URI
to an AWS S3 URI), meltano only writes state
- runs
data is not persisted to S3 at all.
My question - is it possible to configure Meltano to write runs
data to S3 when using S3 as a state backend? We’d like to switch over from Postgres to S3 so we can sunset this particular Postgres database but if that means we lose the runs
records we may need to reconsider, that is valuable data.Matt Menzenski
03/06/2024, 12:11 PMMatt Menzenski
03/06/2024, 12:16 PMEdgar Ramírez (Arch.dev)
03/06/2024, 4:55 PM