proud-pillow-55935
03/11/2021, 2:30 PMsalmon-salesclerk-77709
03/11/2021, 3:07 PMmicroscopic-dream-75195
03/11/2021, 5:41 PM/project
, but that's not the path on your local. What's the specific error you're getting?proud-pillow-55935
03/11/2021, 8:29 PM.meltano/
directory being copied back and forth across the Windows machine and the running container. Especially since I have that in my .dockerignore
file. Maybe the solution is just rebuild the image whenever I make changes, but I was just hoping to be able to make changes and immediately run a pipeline in the UI and it have the most up to date meltano.yml
file.meltano install
in my Dockerfile after copying over the meltano.yml
but still doesn't seem to be working 🤷salmon-salesclerk-77709
03/11/2021, 9:03 PMripe-musician-59933
03/11/2021, 9:05 PMmeltano/meltano
Docker image and mounting your project directory, or are you building a dedicated Docker image for your project?proud-pillow-55935
03/11/2021, 9:06 PMripe-musician-59933
03/11/2021, 9:07 PMdocker run
and everything 🙂salmon-actor-23953
03/11/2021, 9:20 PMproud-pillow-55935
03/11/2021, 9:22 PMripe-musician-59933
03/11/2021, 9:23 PMdocker run
and everything 🙂proud-pillow-55935
03/16/2021, 5:31 PMripe-musician-59933
03/16/2021, 5:32 PMsalmon-actor-23953
03/16/2021, 5:40 PMproud-pillow-55935
03/16/2021, 5:47 PMmeltano add ...
command has to propogate those changes back to the meltano.yml
file on my Windows machine. How would you recommend doing this? I'm currently trying to mount my local directory to the container, but it's running into issues because it's acting like certain Meltano extractors/loaders aren't installed.salmon-actor-23953
03/16/2021, 5:50 PMproud-pillow-55935
03/16/2021, 5:51 PMversion: "3.8"
services:
bw-meltano-ui:
build: .
command: ui
depends_on:
- bw-meltano-db
- bw-meltano-target-db
env_file:
- ./config/meltano-ui.env.dev
# environment:
# - MELTANO_PROJECT_READONLY: 1
ports:
- target: 5000
published: 5000
networks:
- bw-meltano-net
volumes:
- ./meltano.yml:/project/meltano.yml
- ./extract/:/project/extract/
- ./load/:/project/load/
- ./model/:/project/model/
- ./orchestrate/:/project/orchestrate/
- ./transform/:/project/transform/
bw-meltano-target-db:
image: "postgres"
env_file:
- ./config/meltano-target-db.env.dev
networks:
bw-meltano-net:
ports:
- target: 5432
published: 5432
bw-meltano-db:
image: "postgres"
env_file:
- ./config/meltano-db.env.dev
networks:
bw-meltano-net:
ports:
- target: 5432
published: 5433
networks:
bw-meltano-net:
docker-compose.yml
file. Basically, we spin up two databases (one for Meltano, and one for the target-postgres
loader). I originally was binding my entire project directory to the /project
folder on the container, but I thought maybe that that was screwing up everything in the .meltano/
directory since Meltano was letting me know that certain taps/targets weren't installedsalmon-actor-23953
03/16/2021, 5:53 PMvolumes:
section is mapped to individual artifact locations, which could be part of the reason other components may be missed.proud-pillow-55935
03/16/2021, 5:53 PMARG MELTANO_IMAGE=meltano/meltano:latest
FROM $MELTANO_IMAGE
WORKDIR /project
# Install any additional requirements
COPY ./requirements.txt .
RUN pip install -r requirements.txt
# Install all plugins into the `.meltano` directory
COPY ./meltano.yml .
RUN meltano install
# Pin `discovery.yml` manifest by copying cached version to project root
RUN cp -n .meltano/cache/discovery.yml . 2>/dev/null || :
# Copy over remaining project files
COPY . .
# Expose default port used by `meltano ui`
EXPOSE 5000
ENTRYPOINT ["meltano"]
CMD ["ui"]
salmon-actor-23953
03/16/2021, 5:57 PMproud-pillow-55935
03/16/2021, 6:00 PMsalmon-actor-23953
03/16/2021, 7:28 PMDockerfile
. I also proposed we plan to circle back on Windows use cases during next week’s office hours session. @proud-pillow-55935, do keep us posted if this resolves your issue. I’ll create an issue to research that potential Dockerfile issue in the meanwhile. While we’ve got it fresh, do you mind pasting back the updated Dockerfile text here in this thread?proud-pillow-55935
03/16/2021, 7:32 PMARG MELTANO_IMAGE=meltano/meltano:latest
FROM $MELTANO_IMAGE
WORKDIR /project
COPY . .
# Install any additional requirements
RUN pip install -r requirements.txt
# Install all plugins into the `.meltano` directory
RUN meltano install
# Expose default port used by `meltano ui`
EXPOSE 5000
ENTRYPOINT ["meltano"]
CMD ["ui"]
salmon-actor-23953
03/16/2021, 7:45 PM