Guys? Do I need to setup external MELTANO_DATABASE_URI if I am running meltano in Gitlab pipeline? SQLite db is deleted after each run.. Btw: I tried to use Azure Postgresql (

<postgresql://sysadmin>@base-etl-warehouse-postgresql:<mailto:XXXXXX@base-etl-warehousepostgresql.postgres.database.azure.com|XXXXXX@base-etl-warehousepostgresql.postgres.database.azure.com>:5432/meltano?sslmode=require

), but it does not work, due to:

(psycopg2.OperationalError) could not translate host name "<http://base-etl-warehousepostgresql.postgres.database.azure.com|base-etl-warehousepostgresql.postgres.database.azure.com>" to address: Name or service not known

, which is nonsense, because I am using same server ( different db ) for target-postgres, note that Azure is using

sysadmin@base-etl-warehouse-postgresql

format of username

@lukas_mrtvy If you're running in a GitLab pipeline, using an external system database is definitely easiest, although you could also use CI artifacts to store

.meltano/meltano.db

: https://gitlab.com/meltano/meltano/-/merge_requests/1620/diffs#945551220e40ad5fe3602afd73183e0f595abfe5_0_16, or just the state file: https://gitlab.com/meltano/meltano/-/issues/2566 Your Azure PostgresQL URL is likely failing because of that

@

in the username which is interpreted as the username/host separator. If you encode it as

%40

int the URL it should make it to Azure as

@

:

<postgresql://sysadmin%40base-etl-warehouse-postgresql:XXXXXX@base-etl-warehousepostgresql.postgres.database.azure.com:5432/meltano?sslmode=require>

@douwe_maan Thanks. Is OK to have single server ( but different databases ) for ETL and for Meltano itself ?

@lukas_mrtvy Yep, that's fine

Seems that encoding

@

char does not work..Did You ever try to use Azure Postgresql?

Something else to check: Is the hostname

<http://base-etl-warehousepostgresql.postgres.database.azure.com|base-etl-warehousepostgresql.postgres.database.azure.com>

correct or should it be

<http://base-etl-warehouse-postgresql.postgres.database.azure.com|base-etl-warehouse-postgresql.postgres.database.azure.com>

with an extra

-

?

Oh, You are right ! I did not notice. Thanks ( No need to escape

@

)

meltano will not run without state file..

It should be able to! Did you see an error?

I know why, cuz meltano looks for that file relatively to project directory

@lukas_mrtvy Ah, if you specify

--state <path>

on the CLI or

state: <path>

in

meltano.yml

, Meltano will always expect that file to exist. Do you have a way of not passing

--state

when you don't have one yet? Alternatively, you can just write

{}

to a file to serve as an empty state file