https://linen.dev logo
Join Slack
Powered by
hey :wave: invoking a `--custom loader` unsing `ca...
# troubleshooting
f
hey 👋 invoking a 
--custom loader
unsing 
cat hubspotdata |  meltano --log-level=debug invoke pipelinewise-target-bigquery
yields 
Executable 'pipelinewise-target-bigquery' could not be found
but its defined in meltano.yml
Copy code
- name: pipelinewise-target-bigquery
    namespace: pipelinewise_target_bigquery
    pip_url: pipelinewise-target-bigquery
    executable: pipelinewise-target-bigquery
    config:
      credentials_path: xx
it was installed using
Copy code
meltano add --custom loader pipelinewise-target-bigquery
Copy code
(.venv) ⟩meltano install loader pipelinewise-target-bigquery                                                                                                                                                                 
Installing 1 plugins...
Installing loader 'pipelinewise-target-bigquery'...
this is stuck for about 10 minutes now.
even after 
pip install pipelinewise-target-bigquery
its not working
v
meltano install loader pipelinewise-target-bigquery
should do it. Meltano make venvs for each extractor / loader so installing it to your global pip won't help you out here. If the install command is giving you issues we could debug it by setting a log-level and diving in. Another option is to just go remove the folder in 
.meltano/loader/pipelinewise-target-bigquery
and then reinstall
f
.meltano/loader/pipelinewise-target-bigquery
the folder was deleted and successfully reinstalled. the issue persists
i will dive into logs
v
Glad it installed! Good luck 😄
f
ok i got something
logs say:
Copy code
FileNotFoundError: [Errno 2] No such file or directory: '/Users/finn/code//bi/meltano//.meltano/loaders/pipelinewise-target-bigquery/venv/bin/pipelinewise-target-bigquery'
in the folder, there is actually no 
pipelinewise-target-bigquery
only 
target-bigquery
v
Ahh we're onto something!
executable: pipelinewise-target-bigquery
Try changing that to target-bigquery!
f
check, that fixed it
thank you!
v
You're welcome! Good luck :D
f
ran straight into the next issue hahhaha
Copy code
TypeError: an integer is required (got type bytes)
but i guess this is on the connector side
v
haha, progress!
f
rake
e
@finn_frotscher that's an issue with the old version of joblib that target depends on. Running Meltano on Python 3.7 should fix that
f
thank you 🙏
Open in Slack
PreviousNext
Powered by