https://linen.dev logo
Join Slack
Powered by
Alright, so, kind of a weird issue. I've got an sd...
# singer-tap-development
b
Alright, so, kind of a weird issue. I've got an sdk target. It works great locally. It's broken pipes everywhere in a docker container, though. It looks like the 
meltano elt
will try to completely reinstall the target, dependencies and all, and the tap just keeps on trucking, so the pipe is gone by the time the target gets to 
poetry run target-test $* < /dev/stdin
I whipped up a repro case with freshly generated sdk tap and target that is a 
tap-test
that basically yields from a 
range(100)
and a 
target-test
that just writes lines to a text file, just to make sure I wasn't holding it wrong. Console output in thread
message has been deleted
Docker image built with a slightly tweaked dockerfile from 
meltano files add docker
Copy code
ARG MELTANO_IMAGE=meltano/meltano:latest
FROM $MELTANO_IMAGE

WORKDIR /project

# Install any additional requirements
COPY ./requirements.txt . 
RUN pip install -r requirements.txt

# Install all plugins into the `.meltano` directory
COPY ./meltano.yml . 
ADD load/ .
ADD extract/ .
RUN meltano install

# Pin `discovery.yml` manifest by copying cached version to project root
RUN cp -n .meltano/cache/discovery.yml . 2>/dev/null || :

# Don't allow changes to containerized project files
ENV MELTANO_PROJECT_READONLY 1

# Copy over remaining project files
COPY . .

# Expose default port used by `meltano ui`
#EXPOSE 5000

ENTRYPOINT ["meltano"]
a
@brandon_isom - Just want to confirm with you what your desired usage pattern is before advising... Are you wanting to run the tap and target in different containers? Or do you mean when running the EL(T) within a container, pipes are not working correctly? Also - can you provide how you are invoking on the command line (psuedocode is fine) and whether you've tested on your local (to understand if this is docker-related or something else)?
@brandon_isom - Sorry, looking at your attachment now, I see in the log your invocation command line. This looks like a problem with 
poetry install
being inside the target 
.sh
script. I would try removing 
poetry install
from the shell script and making sure poetry install runs in another way - perhaps during within the dockerfile (you'll just need to cd into the directory first).
This is brand new, so my first hunch (as hopefully mitigated above) is a bug/incompatibility in the shell script. That is meant as a helper, modeled after the tap version of the same, but it may be that stdin redirection needs additional tuning to work in this use case.
b
The docker command should be at the top of the log output
docker run --rm repro elt tap-test target-test
a
Yep - sorry. I found that after my first reply. Tahnks.
b
built the image with 
docker build . -t repro
a
Can you try running 
cd /path/to/target && poetry install
within the dockerfile and then removing that install from the target-mytarget.sh shell script?
b
Alright. Dockerfile now looks like this
Copy code
ARG MELTANO_IMAGE=meltano/meltano:latest
FROM $MELTANO_IMAGE

WORKDIR /project

# Install any additional requirements
COPY ./requirements.txt . 
RUN pip install -r requirements.txt

# Install all plugins into the `.meltano` directory
# Copy over remaining project files
COPY . .
RUN cd load/target-test && poetry install
RUN cd extract/tap-test && poetry install

RUN meltano install

# Pin `discovery.yml` manifest by copying cached version to project root
RUN cp -n .meltano/cache/discovery.yml . 2>/dev/null || :

# Don't allow changes to containerized project files
ENV MELTANO_PROJECT_READONLY 1

ENTRYPOINT ["meltano"]
Still getting the broken pipe
Copy code
➜  repro git:(repro) ✗ docker run --rm repro elt tap-test target-test     
meltano     | Running extract & load...
meltano     | No state was found, complete import.
target-test | /project/load/target-test/target-test.sh: 14: /project/load/target-test/target-test.sh: cannot open /dev/stdin: No such device or address
meltano     | Loading failed (2): /project/load/target-test/target-test.sh: 14: /project/load/target-test/target-test.sh: cannot open /dev/stdin: No such device or address
meltano     | ELT could not be completed: Loader failed
ELT could not be completed: Loader failed
a
Thanks for logging this. I still think it is probably a shell script problem. Sorry for the trouble! I'll try to repro on my side and report back.
b
Looks like the poetry install was a red herring, though I'll deffo manually install the taps and targets from now on. I can zip up this repro project if you want
a
I think I'm okay with the info you've provided so far. I guess the one additional question is whether you also tested locally or only via docker. I still am not positive if the container factor is related or if this is just a problem in the script.
b
Yeah works fine locally. Just went into the container interactively, 
meltano elt tap-test target-test
still blows up but 
meltano invoke tap-test | meltano invoke target-test
works fine
a
All the search results are pointing me to docker issues (even when I don't mention docker) so this could indeed be an byproduct of containerization.
Also THANK YOU for confirming it works fine when piped explicitly. That's a big help.
Is this a blocker for you or just an inconvenience? If you are interested in a workaround while we are debugging this, you probably use pipx (or any other means) to fully install your target from it's local folder (or the git repo) and then just use the name of your executable in meltano.yml with no pip_url.
Putting the git repo ref in meltano.yml's 
pip_url
and removing 
executable
(if it matches your use case) would also presumably solve this. (Albeit not ideal for iterative testing.)
b
Ok, its definitely the shell script. If I switch the target to 
pip_url: ./load/target-glue
and then 
RUN cd load/target-test && poetry build
then the 
meltano install
works just fine
And 
docker run --rm repro elt tap-test target-test
works fine
Oh, haha. Great minds think alike I guess
Mostly just an inconvenience. A workaround is fine for now.
a
Thanks, @brandon_isom. I'll log an issue for this and see what we can resolve. Thanks very much for the detailed logs and help in reproducing/diagnosing this issue. It's a big help.
b
Sure thing. Thank you for helping to confirm I hadn't lost my marbles!
Open in Slack
PreviousNext
Powered by