Now I am getting an error when running a pipeline ...
# docker
j
Now I am getting an error when running a pipeline (created by the UI, running as docker run ...) for the first time.
Copy code
File "/project/.meltano/loaders/target-postgres/venv/lib/python3.6/site-packages/psycopg2/__init__.py", line 130, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: Connection refused
	Is the server running on host "localhost" (127.0.0.1) and accepting
	TCP/IP connections on port 5502?
could not connect to server: Cannot assign requested address
	Is the server running on host "localhost" (::1) and accepting
	TCP/IP connections on port 5502?
It looks like postgres should be running ?
```Transform 'tap-carbon-intensity' is missing, trying to install it... Added 'tap-carbon-intensity' to your Meltano project. Installing 'tap-carbon-intensity'... Installed 'tap-carbon-intensity'. Added and installed transforms 'tap-carbon-intensity'. Running extract & load... No state was found, complete import. /project/.meltano/loaders/target-postgres/venv/lib/python3.6/site-packages/psycopg2/__init__.py144 UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: http://initd.org/psycopg/docs/install.html#binary-install-from-pypi. """) Traceback (most recent call last): File "/project/.meltano/loaders/target-postgres/venv/bin/target-postgres", line 11, in <module> load_entry_point('target-postgres==1.1.3', 'console_scripts', 'target-postgres')() File "/project/.meltano/loaders/target-postgres/venv/lib/python3.6/site-packages/target_postgres/__init__.py", line 138, in main state = persist_lines(config, input) File "/project/.meltano/loaders/target-postgres/venv/lib/python3.6/site-packages/target_postgres/__init__.py", line 101, in persist_lines stream_to_sync[stream].create_schema_if_not_exists() File "/project/.meltano/loaders/target-postgres/venv/lib/python3.6/site-packages/target_postgres/db_sync.py", line 297, in create_schema_if_not_exists (schema_name,) File "/project/.meltano/loaders/target-postgres/venv/lib/python3.6/site-packages/target_postgres/db_sync.py", line 133, in query with self.open_connection() as connection: File "/project/.meltano/loaders/target-postgres/venv/lib/python3.6/site-packages/target_postgres/db_sync.py", line 130, in open_connection return psycopg2.connect(conn_string) File "/project/.meltano/loaders/target-postgres/venv/lib/python3.6/site-packages/psycopg2/__init__.py", line 130, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) psycopg2.OperationalError: could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5502? could not connect to server: Cannot assign requested address Is the server running on host "localhost" (::1) and accepting TCP/IP connections on port 5502? INFO GET https://api.carbonintensity.org.uk/regional/intensity/2018-05-15T12:00Z/2018-05-16T12:00Z/regionid/1 Traceback (most recent call last): File "/project/.meltano/extractors/tap-carbon-intensity/venv/bin/tap-carbon-intensity", line 11, in <module> load_entry_point('tap-carbon-intensity==1.0', 'console_scripts', 'tap-carbon-intensity')() File "/project/.meltano/extractors/tap-carbon-intensity/venv/lib/python3.6/site-packages/tap_carbon_intensity/__init__.py", line 131, in main get_data(region_gen) File "/project/.meltano/extractors/tap-carbon-intensity/venv/lib/python3.6/site-packages/tap_carbon_intensity/__init__.py", line 97, in get_data singer.write_record("region", region) File "/project/.meltano/extractors/tap-carbon-intensity/venv/lib/python3.6/site-packages/singer/messages.py", line 237, in write_record time_extracted=time_extracted)) File "/project/.meltano/extractors/tap-carbon-intensity/venv/lib/python3.6/site-packages/singer/messages.py", line 227, in write_message sys.stdout.flush() BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe ELT could not complete, an error happened during the process: Subprocesses didn't exit cleanly: tap(120), target(1) Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/meltano/cli/elt.py", line 70, in elt run_extract_load(elt_context, session, dry_run=dry) File "/usr/local/lib/python3.6/site-packages/meltano/cli/elt.py", line 107, in run_extract_load singer_runner.run(session, **kwa…
here is the complete log of the error.
running v 1.17.1
I see there is an update available to 1.18.0 ; trying that
Same problem.
m
So by default we connect to
localhost:5502
because this is our quickstart setup with the docker-compose. I think the problem here is that we removed the Loader configuration page so you don't see the configuration.
From docker, if you have a
link
with the
warehouse_db
container, then the following ENV should get you running. Alternatively, a
.env
file at the root of your Meltano project could also do the trick.
Copy code
PG_DATABASE: warehouse
PG_PASSWORD: warehouse
PG_USERNAME: warehouse
PG_ADDRESS: warehouse_db
PG_PORT: 5432
PG_SCHEMA: 'tap_postgres'  # IIRC this one is optional
Or you can run the
meltano
container with
--network=host
and it should work out-of-the box.