Hello there! I’m facing once again this strange issue with

MELTANO_STATE_BACKEND_S3_ENDPOINT_URL

— for some reason Meltano cannot find the endpoint url (see the logs bellow). I’ve recreated this behavior on 2 instances with

MELTANO_STATE_BACKEND_S3_ENDPOINT_URL='<https://storage.yandexcloud.net>'

, steps are as follows: • Ubuntu 22.04 LTS • use Python 3.10.12 • install

meltano[s3]==3.10.0

with pipx • run

set -o allexport && source .env && set +o allexport

to export variables from .env • run

meltano --log-level=debug upgrade

to reproduce the issue (persists with taps & targets as well)

2023-10-05T16:29:14.283409Z [debug  ] Looking for endpoint for s3 via: environment_service
2023-10-05T16:29:14.283513Z [debug  ] Looking for endpoint for s3 via: environment_global
2023-10-05T16:29:14.283591Z [debug  ] Looking for endpoint for s3 via: config_service
2023-10-05T16:29:14.283663Z [debug  ] Looking for endpoint for s3 via: config_global
2023-10-05T16:29:14.283734Z [debug  ] No configured endpoint found.

I also have checked if Meltano can accesses the `MELTANO_STATE_BACKEND_S3_ENDPOINT_URL`:

meltano config meltano list | grep ENDPOINT_URL 
2023-10-05T16:36:31.175662Z [info     ] The default environment 'dev' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=<environment name>`.
state_backend.s3.endpoint_url [env: MELTANO_STATE_BACKEND_S3_ENDPOINT_URL] current value: '<https://storage.yandexcloud.net>' (from the MELTANO_STATE_BACKEND_S3_ENDPOINT_URL variable in the environment)

What am I missing here? Would appreciate any help

Those logs come from botocore. I think they just signal that a value couldn't be found in the inspected sources: boto config, instance config,

AWS_ENDPOINT_URL

or

AWS_ENDPOINT_URL_S3

. Meltano sets the endpoint URL directly at client creation, so those sources aren't used if you set

MELTANO_STATE_BACKEND_S3_ENDPOINT_URL

. Are you actually not seeing state files being created in your s3 bucket? Are you using

meltano run

? If that's the case, you can try setting

AWS_ENDPOINT_URL

instead of

MELTANO_STATE_BACKEND_S3_ENDPOINT_URL

.

@edgar_ramirez_mondragon Thank you! Now it works happygoat (I set

AWS_ENDPOINT_URL

instead of

MELTANO_STATE_BACKEND_S3_ENDPOINT_URL

)