Hi, we're trying to upgrade our meltano docker image to

meltano/meltano:v3.3-python3.10

and have a plugin that is dependent on

python3.9

while the rest are compatible with

3.10

. I want to structure the dockerfile so that it supports multiple python versions and do

meltano install

of the plugins during different build stages with different python versions. Can someone show me how the dockerfile should be structured or provide an example? https://docs.meltano.com/reference/settings/#python

@yao_zhang what have you tried so far?

i'm pretty sure i'm doing it wrong, dont have much experience with multistage builds but i have this dockerfile

ARG MELTANO_IMAGE=meltano/meltano:v3.3-python3.9
FROM $MELTANO_IMAGE as build

WORKDIR /build

# Copy over Meltano project directory
COPY meltano.yml .
RUN meltano install extractors tap-orbit
RUN meltano config tap-orbit list

FROM meltano/meltano:v3.3-python3.10
WORKDIR /project
COPY --from=build . .
RUN meltano install extractors tap-airtable tap-pagerduty

# Don't allow changes to containerized project files
ENV MELTANO_PROJECT_READONLY 1

ENTRYPOINT ["meltano"]

the first stage builds successfully but once i move to the second stage it throws an error cause it can't detect the project directory. I thought by doing

COPY --from=build . .

i would be able to copy the existing contents from the first build stage which would include everything (.meltano folder and meltano.yml)?

=> [meltano internal] load build definition from Dockerfile                 0.0s
 => => transferring dockerfile: 791B                                         0.0s
 => [meltano internal] load .dockerignore                                    0.0s
 => => transferring context: 193B                                            0.0s
 => [meltano internal] load metadata for docker.io/meltano/meltano:v3.3-pyt  0.5s
 => [meltano internal] load metadata for docker.io/meltano/meltano:v3.3-pyt  0.5s
 => [meltano stage-1 1/6] FROM docker.io/meltano/meltano:v3.3-python3.10@sh  0.0s
 => [meltano build 1/5] FROM docker.io/meltano/meltano:v3.3-python3.9@sha25  0.0s
 => [meltano internal] load build context                                    0.0s
 => => transferring context: 33B                                             0.0s
 => CACHED [meltano stage-1 2/6] WORKDIR /project                            0.0s
 => CACHED [meltano build 2/5] WORKDIR /build                                0.0s
 => CACHED [meltano build 3/5] COPY meltano.yml .                            0.0s
 => [meltano build 4/5] RUN meltano install extractors tap-orbit             8.2s
 => [meltano build 5/5] RUN meltano config tap-orbit list                    1.3s
 => [meltano stage-1 3/6] COPY --from=build . .                              3.0s
 => ERROR [meltano stage-1 4/6] RUN meltano install extractors tap-airtable  0.8s
------
 > [meltano stage-1 4/6] RUN meltano install extractors tap-airtable tap-pagerduty:
0.665 Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
0.665 join our friendly Slack community.
0.665 
0.665 `meltano install` must be run inside a Meltano project.
0.665 Use `meltano init <project_directory>` to create one.
------
failed to solve: process "/bin/sh -c meltano install extractors tap-airtable tap-pagerduty" did not complete successfully: exit code: 1

Since you are using different workdirs, you likely need to copy from

/build

into

.

(something like

COPY --from=build /build .

)

oh that indeed fix the issue. Thanks @ptd!