I am getting started and trying to load data to Postgres on a brand new EC2 Ubuntu 24.04 using target-postgres. I'm unable to install the target due to the error below. If there is an easier platform to deploy this on, I'll happily switch to that to avoid debugging. I had the same issue with a previous docker-based install on a local dev machine. I am able to install using

uv tool install meltanolabs-target-postgres

but this doesn't seem to be available when running

meltano el ...

which always tries and fails to install target-postgres. Preliminaries

sudo apt update && sudo apt install -y build-essential libpq-dev python3-dev postgresql-client-common
uv tool install meltano

Install target-postgres fails (ERROR: Failed building wheel for psycopg2)

> meltano install loader target-postgres
2024-12-03T20:14:14.130865Z [info     ] Installing 1 plugins
2024-12-03T20:14:14.152560Z [info     ] Installing loader 'target-postgres'
2024-12-03T20:14:20.979307Z [info     ] Logged pip install output to /opt/deployment/meltano/qbsync/.meltano/logs/pip/loaders/target-postgres/pip.log
2024-12-03T20:14:20.979534Z [error    ] Loader 'target-postgres' could not be installed: Failed to install plugin 'target-postgres'.
2024-12-03T20:14:20.979593Z [info     ] Collecting target-postgres
  Using cached target_postgres-1.1.3-py3-none-any.whl.metadata (379 bytes)
Collecting singer-python==5.1.1 (from target-postgres)
  Using cached singer_python-5.1.1-py3-none-any.whl
Collecting psycopg2==2.7.5 (from target-postgres)
  Using cached psycopg2-2.7.5.tar.gz (426 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting inflection==0.3.1 (from target-postgres)
  Using cached inflection-0.3.1-py3-none-any.whl
Collecting pytz==2018.4 (from singer-python==5.1.1->target-postgres)
  Using cached pytz-2018.4-py2.py3-none-any.whl.metadata (20 kB)
Collecting jsonschema==2.6.0 (from singer-python==5.1.1->target-postgres)
  Using cached jsonschema-2.6.0-py2.py3-none-any.whl.metadata (4.6 kB)
Collecting simplejson==3.11.1 (from singer-python==5.1.1->target-postgres)
  Using cached simplejson-3.11.1-cp312-cp312-linux_x86_64.whl
Collecting python-dateutil>=2.6.0 (from singer-python==5.1.1->target-postgres)
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting backoff==1.3.2 (from singer-python==5.1.1->target-postgres)
  Using cached backoff-1.3.2-py3-none-any.whl
Collecting six>=1.5 (from python-dateutil>=2.6.0->singer-python==5.1.1->target-postgres)
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Using cached target_postgres-1.1.3-py3-none-any.whl (6.1 kB)
Using cached jsonschema-2.6.0-py2.py3-none-any.whl (39 kB)
Using cached pytz-2018.4-py2.py3-none-any.whl (510 kB)
Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: psycopg2
  Building wheel for psycopg2 (pyproject.toml): started
  error: subprocess-exited-with-error

  × Building wheel for psycopg2 (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  Building wheel for psycopg2 (pyproject.toml): finished with status 'error'
  ERROR: Failed building wheel for psycopg2
Failed to build psycopg2
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (psycopg2)

Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Failed to install plugin(s)

can you try after enabling the uv venv backend for meltano itself? i.e.

meltano config meltano set venv.backend uv

.

Thanks for the assist. That command worked but now when I try to

meltano install loader target-postgres

I see a bunch of gcc errors:

psycopg/psycopgmodule.c:956:27: error: lvalue required as left operand of assignment
        956 |     Py_TYPE(&lobjectType) = &PyType_Type;
            |                           ^
      psycopg/psycopgmodule.c:993:30: error: lvalue required as left operand of assignment
        993 |     Py_TYPE(&pydatetimeType) = &PyType_Type;
            |                              ^
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1

  help: `psycopg2` (v2.7.5) was included because `target-postgres` (v1.1.3) depends on `psycopg2`

This looks like an issue with psycopg2==2.7.5. When I said "uv works" it does, for 2.9.10, but also fails for 2.7.5: Succeeds:

ubuntu@cf-sync /opt/deployment/meltano/qbsync
> uv tool install psycopg2
Resolved 1 package in 149ms
   Built psycopg2==2.9.10
Prepared 1 package in 6.18s
Installed 1 package in 0.63ms
 + psycopg2==2.9.10
No executables are provided by `psycopg2`

Fails:

> uv tool install psycopg2==2.7.5
Resolved 1 package in 1ms
  × Failed to download and build `psycopg2==2.7.5`
  ╰─▶ Build backend failed to build wheel through `build_wheel` (exit status: 1)

Oh, what's the variant of target-postgres in your

meltano.yml

?

meltanolabs

that's strange then cause the meltanolabs variant depends on

psycopg2-binary

2.9.10

interesting... psycopg = {extras = ["binary"], version = "3.2.3"} psycopg2-binary = "2.9.10"

And I'm seeing

target_postgres-1.1.3-py3-none-any.whl

in the output which is definitely not the meltanolabs variant

I copied some configuration files over to the new server, evidently that is the problem. I'm going to wipe away everything and start over. I appreciate your help.