https://linen.dev logo
Join Slack
Powered by
hi! I just installed a fresh v of meltano in venv ...
# troubleshooting
m
hi! I just installed a fresh v of meltano in venv with py v 3.8 and looks like there some fickleness 🤷‍♀️ pls advise. thx!
d
@mindy_ng What are you trying to accomplish by manually installing jsonschema? Did 
pip install meltano
give you an error?
m
yeah
just starting new project atm
d
Can you share the error you got? In the screenshot, I see you're running 
pip install jsonschema
, but not 
pip install meltano
m
yeah. this is after 
pip3 install meltano
latter half
d
OK, the issue may be that you already had an incompatible version of jsonschema installed in this venv before installing Meltano. Can you try again in a totally new venv?
m
i could but wondering if this is necessary if im just going for simple csv transfer from my github for now
ie skip error and move on to 
meltano init
d
I was initially thinking something was wrong with your venv, but it's possible that some downstream dependency of meltano (e.g. 
hologram
mentioned in the logs) changed its own dependencies to be incompatible with another meltano dependency, 
jsonschema
. So this may be something we have to fix in Meltano anyway...
Let me see if I can find a workaround for you
Sorry for the delay, got pulled into a meeting.
Was this venv clean before you installed 
meltano
? I'm not sure where 
hologram
is coming from, I'm not getting it when I run 
pip install meltano
in a clean venv locally
This is not exactly what you're seeing, but it's possibly related: https://gitlab.com/meltano/meltano/-/issues/2752
I'm looking at that issue now to fix the problem with 
flask
and 
chardet
, but I'm still not sure how 
hologram
factors into this for you. My hunch is that it was already installed in your venv, and meltano is simply incompatible with what you had in the venv before. The recommendation is to always use a clean venv.
Is it possible you've got both dbt and meltano installed in the same venv? The recommendation is to install meltano by itself in a venv, and then install dbt through meltano using 
meltano add transformer dbt
so that it gets its own venv where its dependencies won't conflict with meltano's
m
sorry had lunch, etc. thanks for the update! yeah, i did a venv, installed dbt then tried to use the same venv to install meltano and thats when the errors started popping up
so i was going to deactivate my venv and install meltano and then dbt which is exactly what you just recommended! so here i go
v
for what it's worth I run a separate venv for dbt and meltano just because I haven't figured out the right way to set meltano up to work well with 
meltano invoke dbt run
yet due to configuration issues. I'd bet those can be fixed though 🤷
d
@visch What issues have you been having with 
meltano invoke dbt
?
v
I'll move threads so I don't hijack Mindy!
m
so installed meltano and no errors popped up, but what is wrong with my meltano.yml?
@visch thanks! 😄
d
@mindy_ng I think you need a space between 
-
and 
path
, and then align 
name
etc with another space before those
m
👆 was a success, thank you! sorry for all the q’s. believe me i am scouring through the doc’s as well, but what is my pattern value if i have no subfolder?
d
@mindy_ng Are the CSV or Excel files in your Meltano project root directory? You'll still want a pattern that tells the tap which files to include and exclude, so that it doesn't try to parse 
meltano.yml
for example. Something like 
".*\.csv"
for all CSV files
But it may be easier to move the files into a directory
m
im trying to grab a .csv from my github
i may be misunderstanding u
d
Ah right, my bad, not your local directory
Then 
.*
as the pattern should work
I'm not totally sure tap-spreadsheets-anywhere will understand the blob path you've configured though
m
should i just revert to path ending with .csv then?
and i just re-ran the command after inputting pattern: .* and now its asking for start date
what would that be?
d
It'll use that to filter out files with an older 
modified_at
date, so you can just set it to something older than the files you're using
Which file from https://github.com/mindyng/2021-Projects are you looking for? All files?
m
i revised my path to: 
path: <https://github.com/mindyng/2021-Projects/blob/dbfe45b043e4be16fe58e1fc7082dfe2f0a7781d/onedrop_prod.csv>
d
OK, I think you'll want the URL for the "Raw" button, since that directly downloads the CSV file instead of GitHub's webpage rendering the CSV as a table
m
k. do i still need pattern and start_date defined?
d
I think you do yeah. That seems like it shouldn't be necessary but the tap seems to require it right now anyway
I think it's optimized for directories, locally or over SFTP, or S3 buckets, not individual files by URL
m
gotcha. i reran my last command and no fields errors this time, but ELT could not be run due to “cache_ok” not set to TRUE. ❓ where would i set that?
message has been deleted
im thinking i am veering off to non-meltano land. and if so let me know and ill figure things out
d
The cache_ok stuff can be ignored, I think that's an internal Meltano oversight. Feel free to create an issue for it though so we can clean up the logs 🙂
The error about target-postgres suggests it wasn't installed, does 
meltano install loader target-postgres
fix that?
As for tap-spreadsheets-anywhere, creator @eric_simmerman will be able to provide better support
m
the loader has been installed. i will reach out to eric regarding my error. my csv hasnt been loaded into my postgres. so ill reach out to eric. thanks, douwe!
e
Hey Mindy - per our DMs, your log snapshot doesn’t show any messages produced by tap-spreadsheets-anywhere. If there were any, the name of the tap would print first on the line where ‘meltano’ appears in the lines you’re showing.
In particular that cache message is coming from Meltano core- but I suspect it’s a red herring. You need to determine why target-postgres isn’t being recognized despite the fact that you attempted to install it.
This channel might be able to help diagnose if you shared your current meltano.yml
Also try this
Copy code
meltano config target-postgres list
m
@douwe_maan i just tried another csv import from my github and looks like there is progress, yet still not quite there. wanted to see if you had any advice. here is my updated yml and traceback. think the error is in: 
_psycopg.cpython-38-darwin.so
target-postgres           |   Reason: image not found
d
Sounds like psycopg wasn't installed correctly with all its dependencies yet:https://stackoverflow.com/questions/27264574/import-psycopg2-library-not-loaded-libssl-1-0-0-dylib
It's not exactly the same error message, but clearly related
Open in Slack
PreviousNext
Powered by