Hi, I am just 3-4 days old in Meltano and looking ...
# plugins-general
n
Hi, I am just 3-4 days old in Meltano and looking to do a POC for some plugins but have been getting some errors during plugin installation. For e.g. tap-gitlab is giving me following error - Installing extractor 'tap-gitlab'... Extractor 'tap-gitlab' could not be installed: failed to install plugin 'tap-gitlab'. Running command git clone -q https://gitlab.com/meltano/tap-gitlab.git /tmp/pip-req-build-01qz51be ERROR: Command errored out with exit status 1: command: /home/nitin/nitin/meltano-project/demo-project/.meltano/extractors/tap-gitlab/venv/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-sgv_lai6/ciso8601_ef180008568c44a0b4e6f6e4380d6491/setup.py'"'"'; file='"'"'/tmp/pip-install-sgv_lai6/ciso8601_ef180008568c44a0b4e6f6e4380d6491/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-b3jcgzbh cwd: /tmp/pip-install-sgv_lai6/ciso8601_ef180008568c44a0b4e6f6e4380d6491/ Complete output (18 lines): running bdist_wheel running build running build_py package init file 'ciso8601/__init__.py' not found (or not a regular file) creating build creating build/lib.linux-x86_64-3.8 creating build/lib.linux-x86_64-3.8/ciso8601 copying ciso8601/__init__.pyi -> build/lib.linux-x86_64-3.8/ciso8601 copying ciso8601/py.typed -> build/lib.linux-x86_64-3.8/ciso8601 running build_ext building 'ciso8601' extension creating build/temp.linux-x86_64-3.8 x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DCISO8601_VERSION=2.1.3 -I/home/nitin/nitin/meltano-project/demo-project/.meltano/extractors/tap-gitlab/venv/include -I/usr/include/python3.8 -c module.c -o build/temp.linux-x86_64-3.8/module.o module.c110: fatal error: Python.h: No such file or directory 1 | #include <Python.h> | ^~~~~~~~~~ compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 ---------------------------------------- ERROR: Failed building wheel for ciso8601 ERROR: Command errored out with exit status 1: command: /home/nitin/nitin/meltano-project/demo-project/.meltano/extractors/tap-gitlab/venv/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-sgv_lai6/ciso8601_ef180008568c44a0b4e6f6e4380d6491/setup.py'"'"'; file='"'"'/tmp/pip-install-sgv_lai6/ciso8601_ef180008568c44a0b4e6f6e4380d6491/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-vurqdj3u/install-record.txt --single-version-externally-managed --compile --install-headers /home/nitin/nitin/meltano-project/demo-project/.meltano/extractors/tap-gitlab/venv/include/site/python3.8/ciso8601 cwd: /tmp/pip-install-sgv_lai6/ciso8601_ef180008568c44a0b4e6f6e4380d6491/ Complete output (18 lines): running install running build running build_py package init file 'ciso8601/__init__.py' not found (or not a regular file) creating build creating build/lib.linux-x86_64-3.8 creating build/lib.linux-x86_64-3.8/ciso8601 copying ciso8601/__init__.pyi -> build/lib.linux-x86_64-3.8/ciso8601 copying ciso8601/py.typed -> build/lib.linux-x86_64-3.8/ciso8601 running build_ext building 'ciso8601' extension creating build/temp.linux-x86_64-3.8 x86_64-…
a
Hi Nitin. Looking at the error, I think you are missing gcc. Try 'gcc -v' to see if you have the gnu c compiler, then to install it if you don't 'apt-get install gcc'. Why? One of the dependencies of tap-gitlab has a native dependency. In this case, tap-csv https://gitlab.com/meltano/tap-csv/-/blob/master/setup.py requires
Copy code
install_requires=[
          'singer-python==5.7.0',
          'backoff==1.8.0',
          'requests==2.12.4',
      ],
tap-gitlab https://gitlab.com/meltano/tap-gitlab/-/blob/master/setup.py requires
Copy code
install_requires=[
          'singer-python==5.9.1',
          'requests==2.20.0',
          'strict-rfc3339==0.7',
          'backoff==1.8.0'
      ],
My guess is 'strict-rfc3339==0.7' is the culprit, as that has a dependency on time.
n
gcc -v gives me following output Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper OFFLOAD_TARGET_NAMES=nvptx-none:hsa OFFLOAD_TARGET_DEFAULT=1 Target: x86_64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.3.0-17ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-HskZEa/gcc-9-9.3.0/debian/tmp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
Python dev was missing and running "sudo apt install python3-dev" resolved the issue for me. I am able to install the Snowflake Target as well now. But with tap-mssql I am still getting following error : Installing extractor 'tap-mssql'... Extractor 'tap-mssql' could not be installed: failed to install plugin 'tap-mssql'. ERROR: Could not find a version that satisfies the requirement tap-mssql (from versions: none) ERROR: No matching distribution found for tap-mssql
Are there any known limitations with tap-msql? This is how I am trying to install it - (.venv) nitin@nitin-VirtualBox:~/nitin/meltano-project/demo-project$ meltano add --custom extractor tap-mssql --variant wintersrd Adding new custom extractor with name 'tap-mssql'... (namespace) [tap_mssql]: (pip_url) [tap-mssql]: pip install git+https://github.com/wintersrd/pipelinewise-tap-mssql.git (executable) [pipelinewise-tap-mssql]: (capabilities) [[]]: (settings) [[]]: Extractor 'tap-mssql' already exists in your Meltano project To add it to your project another time so that each can be configured differently, add a new plugin inheriting from the existing one with its own unique name: meltano add extractor tap-mssql--new --inherit-from tap-mssql Installing extractor 'tap-mssql'... Extractor 'tap-mssql' could not be installed: failed to install plugin 'tap-mssql'. ERROR: Could not find a version that satisfies the requirement tap-mssql (from versions: none) ERROR: No matching distribution found for tap-mssql Failed to install plugin(s)
e
Hi @nitin_kharya! For your last comment,
pip_url
should be just either
pipelinewise-tap-mssql
or
git+<https://github.com/wintersrd/pipelinewise-tap-mssql.git>
(i.e. without the
pip install
bit)
n
thanks for pointing that out 🙂 it was not working even after that though ... removed the previous installation with the remove command, reinstalled and then finally ... it worked ! Thanks a lot for the help!!!
e
Ah, makes sense. You had to delete the previously created virtualenv. I'm glad you figured it out 😄