I did a `pipx install meltano==3 0 0` in a dockerfile and th Meltano #troubleshooting

I did a `pipx install meltano==3.0.0` in a dockerf...

josh_lloyd

09/07/2023, 3:23 PM

I did a

pipx install meltano==3.0.0

in a dockerfile and then when I stood up that container and tried to run a pipeline I got:

Copy code

Running command:
meltano run --force tap-gainsightpx target-snowflake
Command pid: 31
Environment 'default' is active
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

No module named 'psycopg2'
Command exited with return code 1

not sure what’s going on here 🤔

mike_ritchie

09/07/2023, 3:28 PM

what’s the full dockerfile?

mike_ritchie

09/07/2023, 3:28 PM

you need to install postgres in docker

mike_ritchie

09/07/2023, 3:29 PM

or change the db URL to point to a postgres db you already have set up

josh_lloyd

09/07/2023, 3:31 PM

dockerfile (up to the point of meltano install)

Copy code

FROM cgr.dev/chainguard/python:latest-dev AS setup
ARG PYTHON_VER=python3.11
ARG DAGSTER_APP=/opt/dagster/app

# Install Python requirements
COPY requirements.txt .
RUN pip install --upgrade pip setuptools
RUN python3 -m pip install --user pipx
RUN python3 -m pipx ensurepath
RUN pip install --no-cache-dir -r requirements.txt
RUN python3 -m pipx install meltano==3.0.0

requirements.txt

Copy code

dbt-snowflake==1.5.3
dagster==1.4.11
dagster_aws
dagster-postgres
dagster-celery[flower,redis,kubernetes]
dagster-k8s
dagster-celery-k8s
dagster-dbt
dagster-meltano @ git+<https://github.com/Widen/dagster-meltano.git@sqlalchemy>

# to resolve errors
pydantic==1.10.11

# to resolve security issues
grpcio>=1.53.0
setuptools>=65.5.1

josh_lloyd

09/07/2023, 3:31 PM

it is supposed to be pointing at a postgres db already … looking into that now

josh_lloyd

09/07/2023, 3:35 PM

yeah, I have a

MELTANO_DATABASE_URI

set in my K8s values.yml file as an env. It was working before when meltano was installed with the rest of my dependencies from the

requirements.txt

file. But I switched to the

pipx

install of meltano because

3.0.0

had conflicting dependencies with my other stuff.

edgar_ramirez_mondragon

09/07/2023, 4:04 PM

@josh_lloyd can you try

pipx install "meltano[psycopg2]==3.0.0"

? https://docs.meltano.com/guide/v3-migration#using-postgres-as-a-backend-now-requires-installing-meltano-with-extra-components

josh_lloyd

09/07/2023, 4:41 PM

yup, that did the trick. Don’t know why I didn’t think of that first … Thank you!

Open in Slack

Previous Next