Hello ! Can someone point me to where can I find information about using a custom tap that comes from a private git repository ?

@juan_sebastian_suarez_valencia - In this case, the best approach I'm aware of would probably be to either pre-install the plugin, or else pre-download the private repo and then try using the local path as the pip_url. From the screenshot above though, it looks like you might already be attempting the second approach. If this is a sdk-based repo and you are wanting to test during development, I opened a WIP MR yesterday for exactly that use case. I'll find the link and post back here in case it is helpful.

Hello @aaronsteers It is not a SDK-based repo unfortunately 😞 And the weird thing is that meltano says that it’s installed and yet it can’t invoke it 🤷 Thanks for having taken the time a saturday to answer my inquiry

@juan_sebastian_suarez_valencia https://meltano.com/docs/plugin-management.html#using-a-custom-fork-of-a-plugin has a section on "If your plugin source is stored in a private repository, you have two options", did you already see that?

Hello @douwe_maan Yes, I tried that unsuccessfully So what I did is that I used poetry to publish my tap to PyPI : https://pypi.org/project/tap-hubspot-meister/ Meltano is able to install it from PyPI [meltano.yml] But I have the same error than when I installed from the local folder Is there a specificity in the folder structure of a tap ? I used the official cookie-cutter of Singer

Can you run

meltano --log-level=debug invoke tap-hubspot-meister

and show us the path it's invoking? Can you verify an executable actually exists at that path?

```.venv) juan_valencia@meltano:~/meltano-hubspot/hubspot-meister$ meltano --log-level=debug invoke tap-hubspot-meister [2021-04-12 155921,780] [19219|MainThread|root] [DEBUG] Creating engine <meltano.core.project.Project object at 0x7f840c06cdf0>@sqlite:////home/juan_valencia/meltano-hubspot/hubspot- meister/.meltano/meltano.db [2021-04-12 155921,864] [19219|MainThread|root] [DEBUG] Created configuration at /home/juan_valencia/meltano-hubspot/hubspot-meister/.meltano/run/tap-hubspot-meister/tap.config.json [2021-04-12 155921,865] [19219|MainThread|root] [DEBUG] Could not find tap.properties.json in /home/juan_valencia/meltano-hubspot/hubspot-meister/.meltano/extractors/tap-hubspot-meis ter/tap.properties.json, skipping. [2021-04-12 155921,865] [19219|MainThread|root] [DEBUG] Could not find tap.properties.cache_key in /home/juan_valencia/meltano-hubspot/hubspot-meister/.meltano/extractors/tap-hubspot -meister/tap.properties.cache_key, skipping. [2021-04-12 155921,865] [19219|MainThread|root] [DEBUG] Could not find state.json in /home/juan_valencia/meltano-hubspot/hubspot-meister/.meltano/extractors/tap-hubspot-meister/state .json, skipping. [2021-04-12 155921,866] [19219|MainThread|root] [DEBUG] Invoking: ['/home/juan_valencia/meltano-hubspot/hubspot-meister/.meltano/extractors/tap-hubspot-meister/venv/bin/tap-hubspot-m eister', '--config', '/home/juan_valencia/meltano-hubspot/hubspot-meister/.meltano/run/tap-hubspot-meister/tap.config.json', '--discover'] [2021-04-12 155921,866] [19219|MainThread|root] [DEBUG] Env: {'SHELL': '/bin/bash', 'SSH_AUTH_SOCK': '/tmp/ssh-xOF7gkiG6F/agent.18978', 'PWD': '/home/juan_valencia/meltano-hubspot/hu bspot-meister', 'LOGNAME': 'juan_valencia', 'XDG_SESSION_TYPE': 'tty', 'MOTD_SHOWN': 'pam', 'HOME': '/home/juan_valencia', 'LANG': 'C.UTF-8', 'LS_COLORS': 'rs=0di=01;34ln=01;36:mh=00 pi=40;33so=01;35do=01;35bd=40;33;01cd=40;33;01or=40;31;01mi=00su=37;41sg=30;43ca=30;41tw=30;42ow=34;42st=37;44ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.t az=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31 :*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.al z=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif =01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=0 1;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;3 5:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.a u=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:', 'V IRTUAL_ENV': '/home/juan_valencia/meltano-hubspot/hubspot-meister/.meltano/extractors/tap-hubspot-meister/venv', 'SSH_CONNECTION': '74.125.73.97 50390 10.156.0.4 22', 'LESSCLOSE': '/us r/bin/lesspipe %s %s', 'XDG_SESSION_CLASS': 'user', 'TERM': 'xterm-256color', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'USER': 'juan_valencia', 'SHLVL': '1', 'XDG_SESSION_ID': '41', 'XDG_ RUNTIME_DIR': '/run/user/1002', 'PS1': '(.venv) \\[\\e]0;\\u@\\h: \\w\\a\\]${debian_chroot:+($debian_chroot)}\\[\\033[01;32m\\]\\u@\\h\\[\\033[00m\\]:\\[\\033[01;34m\\]\\w\\[\\033[00m\ \]\\$ ', 'SSH_CLIENT': '74.125.73.97 50390 22', 'XDG_DATA_DIRS': '/usr/local/share/usr/share/v…

OK. What do you see when you run

meltano install extractor tap-hubspot-meister

?

message has been deleted

@juan_sebastian_suarez_valencia I think the issue is that your

setup.py

doesn't define any executables

Yes

You'll want to keep only one

Yes, I know but since with poetry it was much easier to test the deployment I installed it today

When I choose "Download files" on https://pypi.org/project/tap-hubspot-meister/, the

setup.py

I get is missing that

entry_points

section which is necessary to determine the executable:

# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['tap_hubspot_meister', 'tap_hubspot_meister.test']

package_data = \
{'': ['*'], 'tap_hubspot_meister': ['schemas/*']}

install_requires = \
['hubspot-api-client>=3.7.2,<4.0.0',
 'pydash>=5.0.0,<6.0.0',
 'rich>=10.1.0,<11.0.0',
 'singer-python>=5.12.1,<6.0.0',
 'singer>=0.1.1,<0.2.0']

setup_kwargs = {
    'name': 'tap-hubspot-meister',
    'version': '0.1.0',
    'description': '',
    'long_description': None,
    'author': 'Juan Sebastian Suarez Valencia',
    'author_email': 'juan.valencia@meisterlabs.com',
    'maintainer': None,
    'maintainer_email': None,
    'url': None,
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'python_requires': '>=3.7,<4.0',
}


setup(**setup_kwargs)

Ok that would explain why it’s not working now with Pypi But last week I was using only setup.py and I had the same problem. Meaning, that the manual installation was not working with Meltano

Matching the

tap-hubspot-meister=tap_hubspot_meister:main

you have in

setup.py

Ok I’ll try that

Interesting, I think the underlying issues may have been different, but of course we can't debug what you had last week anymore 🙂

haha correct 🙆

😄