Hi, meltano just released

v3.0

that we're trying to upgrade to from v2. I am following the guide and would like some help around the section of using postgres as a system state backend. we have an existing postgres instance and use the docker image of meltano. I've tried adding

psycopg2==2.9.7

to the requirements file here but i get this error block:

> [meltano 4/6] RUN pip install -r requirements.txt:
0.743 Collecting psycopg2==2.9.7 (from -r requirements.txt (line 2))
0.888   Downloading psycopg2-2.9.7.tar.gz (383 kB)
0.942      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 383.5/383.5 kB 7.6 MB/s eta 0:00:00
1.025   Preparing metadata (setup.py): started
1.200   Preparing metadata (setup.py): finished with status 'error'
1.204   error: subprocess-exited-with-error
1.204   
1.204   × python setup.py egg_info did not run successfully.
1.204   │ exit code: 1
1.204   ╰─> [23 lines of output]
1.204       running egg_info
1.204       creating /tmp/pip-pip-egg-info-k0ldmwnq/psycopg2.egg-info
1.204       writing /tmp/pip-pip-egg-info-k0ldmwnq/psycopg2.egg-info/PKG-INFO
1.204       writing dependency_links to /tmp/pip-pip-egg-info-k0ldmwnq/psycopg2.egg-info/dependency_links.txt
1.204       writing top-level names to /tmp/pip-pip-egg-info-k0ldmwnq/psycopg2.egg-info/top_level.txt
1.204       writing manifest file '/tmp/pip-pip-egg-info-k0ldmwnq/psycopg2.egg-info/SOURCES.txt'
1.204       
1.204       Error: pg_config executable not found.
1.204       
1.204       pg_config is required to build psycopg2 from source.  Please add the directory
1.204       containing pg_config to the $PATH or specify the full executable path with the
1.204       option:
1.204       
1.204           python setup.py build_ext --pg-config /path/to/pg_config build ...
1.204       
1.204       or with the pg_config option in 'setup.cfg'.
1.204       
1.204       If you prefer to avoid building psycopg2 from source, please install the PyPI
1.204       'psycopg2-binary' package instead.
1.204       
1.204       For further information please check the 'doc/src/install.rst' file (also at
1.204       <<https://www.psycopg.org/docs/install.html>>).
1.204       
1.204       [end of output]
1.204   
1.204   note: This error originates from a subprocess, and is likely not a problem with pip.
1.206 error: metadata-generation-failed
1.206 
1.206 × Encountered error while generating package metadata.
1.206 ╰─> See above for output.
1.206 
1.206 note: This is an issue with the package mentioned above, not pip.
1.206 hint: See above for details.
------
failed to solve: process "/bin/sh -c pip install -r requirements.txt" did not complete successfully: exit code: 1

is there a preferred meltano recommended way of building the library or would using

pip install psycopg2-binary

suffice for quick install?

Meltano's official Docker image is build off of this Dockerfile: https://github.com/meltano/meltano/blob/main/docker/meltano/Dockerfile In that Dockerfile, we run

pip install "meltano[azure,gcs,mssql,postgres,psycopg2,s3]==${MELTANO_VERSION}"

So it should already have the required extras installed. What issue are you running into with the official v3 Docker images @yao_zhang? You shouldn't need to install any Python packages yourself for Meltano. Doing so is not recommended, as having other packages installed into Meltano's virtual environment is a common source of issues.

oh ok good to know! i wasn't sure if this library was already included so just went along with the migration doc. but is it safe to say that if we're using the official docker image that there isn't anything additional things we need to do other than making sure there's backwards compatibility with v2 and the breaking changes here?

That should be the case, but please let us know if you run into any trouble