Greetings. It seems I can not just pip install me...
# troubleshooting
a
Greetings. It seems I can not just pip install meltano on amazon linux.
Copy code
(.venv) [srvc_singer@ip-10-0-0-211 lab49-singer-runtime]$ python --version
Python 3.8.2
(.venv) [srvc_singer@ip-10-0-0-211 lab49-singer-runtime]$ cat /etc/os-release
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="<https://amazonlinux.com/>"
I use the version
pip install meltano==1.77.0
locally on my mac just fine I can’t use any version on the fresh AWS ec2 instance in its own clean .venv even when I execute
(.venv) [srvc_singer@ip-10-0-0-211 lab49-singer-runtime]$ pip install meltano
The end of the output looks like:
Copy code
The conflict is caused by:
    meltano 1.78.0 depends on watchdog<0.10.0 and >=0.9.0
    meltano 1.77.0 depends on watchdog<0.10.0 and >=0.9.0
    meltano 1.76.0 depends on watchdog<0.10.0 and >=0.9.0
    meltano 1.75.0 depends on watchdog<0.10.0 and >=0.9.0
    meltano 1.74.0 depends on watchdog<0.10.0 and >=0.9.0
    meltano 1.73.0 depends on watchdog<0.10.0 and >=0.9.0
    meltano 1.72.0 depends on watchdog<0.10.0 and >=0.9.0
    meltano 1.71.0 depends on watchdog<0.10.0 and >=0.9.0
    meltano 1.70.0 depends on watchdog<0.10.0 and >=0.9.0
    meltano 1.69.0 depends on watchdog<0.10.0 and >=0.9.0
    meltano 1.68.0 depends on watchdog<0.10.0 and >=0.9.0
Note: after I
yum install libffi-devel
,
yum install sqlite-devel
and rebuilt python 3.8 then watchdog would install and meltano would install and execute. Once I move from sqlite to rds I wouldn’t need
sqlite-devel
.
d
I haven’t run into this specific issue, but when I’ve had similar issues upgrading pip has been helpful:
Copy code
pip install --upgrade pip
pip install meltano
a
That wasn’t enough for AWS linux. I always follow the meltano docs and use a clean vitrual environment.
Copy code
python3.8 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install meltano==1.77.0
However meltano wouldn’t install on a clean python 3.8.2 install without adding those extra libraries at the OS level before compiling python. What distro’s of linux are you using? Something with more stuff already in it by default? For instance the meltano build assumes SQLite is already part of the OS. you must be using a distro with SQLite on it. Watchdog depends on OS components that are not mandatory for python or linux. Upgrading pip doesn’t add OS components that watchdog needs. Especially ones that need to be present before python is compiled.
I suspect this is true for other versions of python since this is really a requirement of the OS.
I noticed Meltano actually installs without SQLite in the OS it just wont run successfully in the default config.