Hey all, I'm attempting to initialize a meltano project using both an S3 bucket as the State Backend and an RDS Postgres instance for the Database. When installing meltano on my EC2 instance I use the following to get the required optional components:

pip3 install -q meltano[s3]
pip3 install -q meltano[postgres]

I set the following environment variables:

MELTANO_DATABASE_URI=postgresql+psycopg://<username>:<password>@<host>:<port>/<database>
MELTANO_STATE_BACKEND_URI=s3://<aws_access_key_id>:<aws_secret_access_key>@<your bucket name>/<prefix for state JSON blobs>

MELTANO_DATABASE_URI Specification for PostgreSQL MELTANO_STATE_BACKEND_URI Specification for S3 When running

meltano init project

I get the following error:

Creating system database...Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Invalid IPv6 URL

Has anyone else run into this issue or is there a CLI flag I'm missing when initing the project?

@patrick_travis can you try un-setting the env vars and running

meltano init project

again. You can set the state backend and system db after that and check their values with

meltano config meltano list

.

I figured it out. When passing in the RDS password, there were reserved characters being set, which aren't allowed in the credential part of a URL. This didn't seem to be an issue with

psql

(which I was also using in my setup script) or when I hard coded the URL in python to test with

SQLAlchemy

, but it seems as though

meltano

isn't able to handle the reserved characters when set in it's config. I was about to get around this by mapping the special characters to their % encoded value for

meltano

like so:

url_encode() {
    local length="${#1}"
    for (( i = 0; i < length; i++ )); do
        local c="${1:i:1}"
        case $c in
            [a-zA-Z0-9.~_-]) printf "$c" ;;
            *) printf '%%%02X' "'$c" ;;
        esac
    done
}

ENCODED_RDS_PASSWORD=$(url_encode "${RDS_PASSWORD}")