Hey everyone.
During the local development I want to have a different state_backend than the production.
• When I hardcode the s3 url (A) it works great.
• When I am passing through env variable the MELTANO_ENVIRONMENT, it doesn't work. (B).
Through the logs I am able to see the passed value of the MELTANO_ENVIRONMENT, the process is able to write into the right path of s3, to create empty tables in my destination (Snowflake), but it doesn't fill them up.
It feels that some part of the meltano project can correctly read the variable and the full s3 path, and in other parts it is unable to locate it and thus failing to insert new entries.
• Is this a known issue?
A)
state_backend:
uri: <s3://el-meltano-state-bucket/dev>
B)
state_backend:
uri: <s3://el-meltano-state-bucket/$MELTANO_ENVIRONMENT>