me again! This one is interesting So i have melta...
# troubleshooting
d
me again! This one is interesting So i have meltano containerized, and my docker file contains applicable lines of the below (this is not the whole dockerfile, just things i think are directly applicable)
Copy code
USER root
ENV MELTANO_FOLDER=${AIRFLOW_HOME}/meltano
ENV MELTANO_PROJECT_ROOT=${MELTANO_FOLDER}

RUN mkdir -p ${MELTANO_FOLDER}

RUN apt-get install python3-venv
RUN apt-get autoremove -yqq --purge \
    && apt-get clean

COPY /meltano/meltano.yml ${MELTANO_FOLDER}
COPY /meltano/meltano_entrypoint.sh ${MELTANO_FOLDER}
COPY /meltano/plugins ${MELTANO_FOLDER}/plugins

RUN chown -R airflow: ${MELTANO_FOLDER}

USER airflow
RUN pip install --upgrade pip
RUN pip install -r /requirements.txt
RUN chmod +rwx ${MELTANO_FOLDER}
My docker compose has the following entrypoint (some of these lines are for debugging, like ls -a)
Copy code
#!/usr/bin/env bash

set -e

cd meltano
pip install --upgrade pip
ls -a
mkdir .meltano/
meltano install
meltano ui --bind-port=4242
I am getting the following error on the install command
Copy code
grasshopper-meltano     | Loader 'target-snowflake' could not be installed: failed to upgrade pip to the latest version.
grasshopper-meltano     | ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
grasshopper-meltano     | WARNING: You are using pip version 22.0.4; however, version 22.1.2 is available.
grasshopper-meltano     | You should consider upgrading via the '/opt/airflow/meltano/.meltano/loaders/target-snowflake/venv/bin/python -m pip install --upgrade pip' command.
Does anyone have any thoughts there?
not to be a PITA but this is one of the few things blocking me from deploying this at production level. Any help would be greatly appreciated
so it looks like my issue was that i was extending airflows docker image, and meltano really wants to run as root while airflow really does not. I was able to move forward by building my own airflow docker image from scratch instead