michael_cooper
08/24/2020, 9:43 PMNo such file or directory: '/projects/.meltano/extractors/tap-github/venv/bin/tap-github': '/projects/.meltano/extractors/tap-github/venv/bin/tap-github'
error despite it being within my project directory. I am using the Meltano Docker image and running on Ubuntu if that helps.douwe_maan
08/24/2020, 10:14 PM.meltano
(https://gitlab.com/meltano/meltano/-/issues/2223), but that's supposed to happen automatically as part of meltano add
(https://meltano.com/docs/command-line-interface.html#add). 😕
Did you see anything in your meltano add
output that suggested something went wrong?
Does the entire .meltano/extractors/tap-github/venv/bin/tap-github
path not exist, or is only the executable missing, for example?
You can explicitly install the tap using meltano install extractor tap-github
, which should solve your problem, but I'm not sure how you got into this situation 😬 Running meltano install
is usually only necessary after cloning/pulling a project repo, because then the .meltano
directory will be missing.michael_cooper
08/25/2020, 3:49 PM.meltano/extractors/tap-github/venv/bin/tap-github
and I did use meltano install extractor --custom tap-github
to install it initially. Since it was a singer tap not officially supported by Meltano, I assume I need to use the --custom
flag?douwe_maan
08/25/2020, 3:50 PMmeltano install
doesn't need a --custom
flag, since it will install all plugins defined in meltano.yml
, where you've already added tap-github
🙂douwe_maan
08/25/2020, 3:51 PM.meltano/extractors/tap-github/venv/bin/tap-github
already existed before you ran meltano install
, despite the No such file or directory: '/projects/.meltano/extractors/tap-github/venv/bin/tap-github'
error message?michael_cooper
08/25/2020, 3:59 PMmeltano install
and followed the interactive instructions. I got all the way through the process and got a successful installation.douwe_maan
08/25/2020, 4:00 PMmeltano add --custom
, not meltano install
, right?michael_cooper
08/25/2020, 4:00 PMmeltano elt
or meltano invoke
with the tap-github
plugin and got the errors.michael_cooper
08/25/2020, 4:00 PMdouwe_maan
08/25/2020, 4:00 PMmeltano install extractor tap-github
, or are elt and invoke working now?michael_cooper
08/25/2020, 4:01 PMdouwe_maan
08/25/2020, 4:01 PMmichael_cooper
08/25/2020, 5:04 PMextractor/tap-github
and removed references from the meltano.yml
file to start over. Ran meltano add extractor --custom tap-github
to go through the process of installing, which got me a successful installation. Then ran meltano install extractor tap-github
which then throws this error:
FileNotFoundError: [Errno 2] No such file or directory: '/projects/.meltano/extractors/tap-github/venv/bin/pip': '/projects/.meltano/extractors/tap-github/venv/bin/pip'
Both .meltano/extractors/tap-gitub/venv/bin/pip
and ``.meltano/extractors/tap-gitub/venv/bin/tap-github` exists in my directory.douwe_maan
08/25/2020, 5:19 PMmichael_cooper
08/25/2020, 6:27 PMGithub/learn_meltano
2. run docker pull meltano/meltano
3. run docker run -v $(pwd):/projects -w /projects meltano/meltano init learn_meltano
4. cd learn_meltano
5. sudo docker run --interactive -v $(pwd):/project -w /project meltano/meltano add --custom extractor tap-github
6. sudo docker run -v $(pwd):/projects -w /projects meltano/meltano install extractor tap-github
douwe_maan
08/25/2020, 7:00 PM/projects
in the paths where I was expecting /project
, but as long as you're consistently mounting the directory and setting the working directory, you should be good.
That leaves me quite confused, since if .meltano/extractors/tap-gitub/venv/bin/pip
exists inside the working directory you're mounting and telling Docker to use as the container's working directory, then why wouldn't the file exist there from the container's perspective as well?douwe_maan
08/25/2020, 7:02 PM/projects/.meltano/extractors/tap-gitub/venv/bin/tap-github
that way? And then try running meltano invoke tap-github
from inside that bash shell?michael_cooper
08/25/2020, 7:20 PMsudo docker run -it --entrypoint bash meltano/meltano
gets me into the /projects
, which contains nothing.michael_cooper
08/25/2020, 7:21 PMdouwe_maan
08/25/2020, 7:22 PM-v $(pwd):/projects -w /projects
😉douwe_maan
08/25/2020, 7:24 PMmichael_cooper
08/25/2020, 7:24 PMdouwe_maan
08/25/2020, 7:26 PMdouwe_maan
08/25/2020, 7:51 PM/project
, but then elt
and invoke
were run with the project directory mounted at /projects
instead, which caused the plugin executables inside the plugin-specific Python virtual environments to fail because they contained hard-coded absolute paths that assumed the project still lived at /project
.