https://linen.dev logo
Join Slack
Powered by
Hello everyone, I am running into a bug when testi...
# troubleshooting
j
Hello everyone, I am running into a bug when testing my newly created tap with the Meltano SDK. (Meltano version 1.87.1, Meltano SDK version 0.3.13 inside a Debian Docker container) Everything works fine when I manually use Meltano to invoke both the tap and the target:
Copy code
meltano invoke tap-twitter | target-s3-parquet
However, things break when I use the Meltano elt command:
Copy code
meltano elt tap-twitter target-s3-parquet
It then throws this error:
Copy code
/target-s3-parquet/target-s3-parquet.sh: 14: cannot open /dev/stdin: No such device or address
I cannot really figure out why it throws this error. I think it has something to do with the shell file it executes to run the target.
Hopefully this is the right channel, I am not sure if it has something to do with Meltano or the SDK.
e
hi @jules_huisman ! I'm not familiar with that target implementation. What are the contents of 
target-s3-parquet.sh
?
j
Hi Edgar, it contains the starting script from the target cookiecutter:
Copy code
#!/bin/sh

# This simple script allows you to test your target from any directory, while still taking
# advantage of the poetry-managed virtual environment.
# Adapted from: <https://github.com/python-poetry/poetry/issues/2179#issuecomment-668815276>

unset VIRTUAL_ENV

STARTDIR=$(pwd)
TOML_DIR=$(dirname "$0")

cd "$TOML_DIR" || exit
poetry install 1>&2
poetry run target-s3-parquet $* < /dev/stdin
I think I "solved" the problem. If I simply remove 
< /dev/stdin
everything works fine. I am not entirely sure what this part does.
Copy code
...
poetry run target-s3-parquet $*
e
Oh so this is an SDK-based target. Nice!
I am not entirely sure what this part does.
The 
$*
part passes any arguments from the .sh script invocation like 
--config
, etc. to the command 
poetry run target-s3-parquet
If I simply remove 
< /dev/stdin
 everything works fine.
This may be a bug in the cookiecutter shell script. cc @aaronsteers
j
Ah yes, makes sense!
It only breaks when running with 
meltano elt
it works with any other invocation. Maybe that helps with debugging.
a
Very helpful - thanks, @jules_huisman!
n
I also just ran into this. Is there a workaround that isn't just remove 
< /dev/stdin
? I'm encountering this when running on GH actions.
It works completely fine when I run locally.
Open in Slack
PreviousNext
Powered by