Hello, Impossible to install dbt-postgres utility :

Utility 'dbt-postgres' could not be installed: Failed to install plugin 'dbt-postgres'.
Collecting git+<https://github.com/meltano/dbt-ext.git@main>
  Cloning <https://github.com/meltano/dbt-ext.git> (to revision main) to /tmp/pip-req-build-bxr10h5s
  Running command git clone --filter=blob:none --quiet <https://github.com/meltano/dbt-ext.git> /tmp/pip-req-build-bxr10h5s
  Resolved <https://github.com/meltano/dbt-ext.git> to commit 2e55a1e00bf3611978a5253fef92ef62b85788fe
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting dbt-core
  Downloading dbt_core-1.8.2-py3-none-any.whl.metadata (3.9 kB)
Collecting dbt-postgres
  Downloading dbt_postgres-1.8.1-py3-none-any.whl.metadata (3.4 kB)
Collecting agate<1.10,>=1.7.0 (from dbt-core)
  Downloading agate-1.9.1-py2.py3-none-any.whl.metadata (3.2 kB)
Collecting Jinja2<4,>=3.1.3 (from dbt-core)
  Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting mashumaro<4.0,>=3.9 (from mashumaro[msgpack]<4.0,>=3.9->dbt-core)
  Downloading mashumaro-3.13.1-py3-none-any.whl.metadata (114 kB)
     ââââââââââââââââââââââââââââââââââââââââ 114.3/114.3 kB 6.1 MB/s eta 0:00:00
Collecting logbook<1.6,>=1.5 (from dbt-core)
  Downloading Logbook-1.5.3.tar.gz (85 kB)
     ââââââââââââââââââââââââââââââââââââââââ 85.8/85.8 kB 98.3 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting click<9.0,>=8.0.2 (from dbt-core)
  Downloading click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting networkx<4.0,>=2.3 (from dbt-core)
  Downloading networkx-3.2.1-py3-none-any.whl.metadata (5.2 kB)
Collecting protobuf<5,>=4.0.0 (from dbt-core)
  Downloading protobuf-4.25.3-cp37-abi3-manylinux2014_x86_64.whl.metadata (541 bytes)
Collecting requests<3.0.0 (from dbt-core)
  Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting pathspec<0.13,>=0.9 (from dbt-core)
  Downloading pathspec-0.12.1-py3-none-any.whl.metadata (21 kB)
Collecting sqlparse<0.6.0,>=0.5.0 (from dbt-core)
  Downloading sqlparse-0.5.0-py3-none-any.whl.metadata (3.9 kB)
Collecting dbt-extractor<=0.6,>=0.5.0 (from dbt-core)
  Downloading dbt_extractor-0.5.1-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.2 kB)
Collecting minimal-snowplow-tracker<0.1,>=0.0.2 (from dbt-core)
  Downloading minimal-snowplow-tracker-0.0.2.tar.gz (12 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting dbt-semantic-interfaces<0.6,>=0.5.1 (from dbt-core)
  Downloading dbt_semantic_interfaces-0.5.1-py3-none-any.whl.metadata (2.6 kB)
Collecting dbt-common<2.0,>=1.0.4 (from dbt-core)
  Downloading dbt_common-1.3.0-py3-none-any.whl.metadata (5.3 kB)
Collecting dbt-adapters<2.0,>=1.1.1 (from dbt-core)
  Downloading dbt_adapters-1.2.1-py3-none-any.whl.metadata (2.5 kB)
Collecting packaging>20.9 (from dbt-core)
  Downloading packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting pytz>=2015.7 (from dbt-core)
  Downloading pytz-2024.1-py2.py3-none-any.whl.metadata (22 kB)
Collecting pyyaml>=6.0 (from dbt-core)
  Downloading PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting daff>=1.3.46 (from dbt-core)
  Downloading daff-1.3.46.tar.gz (149 kB)
     ââââââââââââââââââââââââââââââââââââââââ 149.8/149.8 kB 61.3 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting typing-extensions>=4.4 (from dbt-core)
  Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting psycopg2<3.0,>=2.9 (from dbt-postgres)
  Downloading psycopg2-2.9.9.tar.gz (384 kB)
     ââââââââââââââââââââââââââââââââââââââââ 384.9/384.9 kB 49.7 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  error: subprocess-exited-with-error
  
  Ã python setup.py egg_info did not run successfully.
  â exit code: 1
  â°â> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Preparing metadata (setup.py): finished with status 'error'
error: metadata-generation-failed

à Encountered error while generating package metadata.
â°â> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Failed to install plugin(s)

What system are you running? I would check for compatibility issues with

psyopg2

, which is a bit notorious for that kind of thing. In a fresh virtual environment, you can try just installing that package and see if you get a more detailed error:

cd /tmp
python3 -m venv .venv
. .venv/bin/activate
pip install psycopg2==2.9.9

# cleanup after
deactivate
rm -rf .venv

No issue with the removal; it's just warning you about some deprecated setting kinds the plugin definition is still using (unrelated).

Hi @Reuben (Matatika) , I'm on your docker image : meltano/meltano:v3.3-python3.9

Yeah, so that image doesn't have

pg_config

available to build

psycopg2

from source. Installing

dbt-postgres

here assumes the image is a production environment purely because it is Linux-based and provides

psycopg2

over

psycopg2-binary

- the latter would require no build and circumvent the error you are seeing. https://github.com/dbt-labs/dbt-postgres/blob/007e8b8221b95e35c337de4a21300e235af68790/pyproject.toml#L26-L29 I would try changing the plugin

pip_url

to

dbt-core psycopg2-binary==2.9.9 git+<https://github.com/meltano/dbt-ext.git@main>

and go from there. If that doesn't work or it fails down the line, add these other dependencies to the

pip_url

and try again. The other option is to extend the Meltano image in your own Dockerfile and install the

psycopg2

build prerequisites.

I recently added a note to the getting started guide. Something like this should work:

utilities:
  - name: dbt-postgres
    pip_url: dbt-core dbt-postgres<1.8 git+<https://github.com/meltano/dbt-ext.git@main>

So