I'm trying to figure out how I can use completely different state backends for different environments. Eg. I want to use

gcs

for production and local file system for local development. I have so far only found solutions involving

MELTANO_ENVIRONMENT

variable, but can't seem to figure out how that lets me change anything but the path/bucket name in the same backend. I've tried putting

state_backend

under an environment like so, but that seems to ignore the setting

environments:
- name: prod
  config:
    state_backend: ...

I set env variables in prod only to achieve this on azure, there's nothing in my

meltano.yml

regarding state

MELTANO_STATE_BACKEND_AZURE_CONNECTION_STRING

and

MELTANO_STATE_BACKEND_URI

are the two variables I set when running 'non-locally'. That gives me a sqlite backend locally (default behaviour) and Azure Storage blob backend on the cloud. Docs on env variables: https://docs.meltano.com/reference/settings/#state-backends

@Andy Carter thanks for the tip with env vars, that might do the trick... Yeah I've consulted https://docs.meltano.com/concepts/state_backends but it leaves open whether that supports environment configs in meltano.yml (it's also a rather top level configuration compared to plugins...). That's why I mentioned I tried it, but it didn't work. Maybe my version of meltano is also too old to support it (I'm still on 2.12)

Are you thinking of setting the state backend on a per tap-target basis?