On the private taps question, is anyone using Azure Repos as a location rather than git? When I try to pass a

pip_url

like:

<mailto:ssh+git@ssh.dev.azure.com|ssh+git@ssh.dev.azure.com>:v3/my_org/my_team/tap-privatetap

I get a 'looks like a path' error:

ERROR: Invalid requirement: '<mailto:ssh+git@ssh.dev.azure.com|ssh+git@ssh.dev.azure.com>:v3/myorg/myteam/tap-privatetap'
Hint: It looks like a path. File '<mailto:ssh+git@ssh.dev.azure.com|ssh+git@ssh.dev.azure.com>:v3/myorg/myteam/tap-privatetap' does not exist.

I definitely have working ssh access as using it in vscode already for git with that repo.

Doesn't look like anything to do with azure. The path is wrong https://docs.meltano.com/guide/plugin-management#using-a-custom-fork-of-a-plugin:~:text=Authentic[…]tead%2C%20and%20specify%20a%20git%2Bssh%20URL%3A Is an example for ssh

pip_url: <git+ssh://git@gitlab.com/meltano/tap-gitlab.git>

I copied the url from the

Git Clone

command in Azure repos, assumed that would work. Will take a look

You may have

ssh+git

backwards, can you try

git+ssh

? https://pip.pypa.io/en/stable/topics/vcs-support/#supported-vcs

Thanks for the tip, I have tried both variants and no luck. I think the issue might be related to having whitespace in the name.

Running command git clone --filter=blob:none --quiet 'ssh://****@ssh.dev.azure.com:v3/myorg/my%20team/tap-mailchimp' /tmp/pip-req-build-v_608p9j
  ssh: Could not resolve hostname <http://ssh.dev.azure.com:v3|ssh.dev.azure.com:v3>: Name or service not known
  fatal: Could not read from remote repository.

If I remove

:v3

from the URL the error becomes

Running command git clone --filter=blob:none --quiet 'ssh://****@ssh.dev.azure.com/myorg/my%20team/tap-mailchimp' /tmp/pip-req-build-vyz9k6gt
  remote: Command git-upload-pack '/myorg/my team/tap-mailchimp' is not in expected format.
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  error: subprocess-exited-with-error

I'll try and get the whitespace removed and see if that improves things

Can you share the exact related pip urls for each? Also ideally in the same cmd prompt could you share you successfully git cloning the repo locally

- name: tap-mailchimp
    pip_url: <git+ssh://git@ssh.dev.azure.com>:v3/EmpiricPowerApps/Sales%20and%20Marketing/tap-mailchimp
    config:
      start_date: 2013-01-01
      api_key: ${TAP_MAILCHIMP_API_KEY}
      dc: ${TAP_MAILCHIMP_DC}

results in:

(empiric-elt-py3.10) andycarter@DESKTOP-D6ERPSP:~/vscode_projects/empiric_elt/empiric_meltano$ meltano --log-level=debug install extractor tap-mailchimp --clean
2023-04-19T08:10:42.618967Z [debug    ] Creating engine '<meltano.core.project.Project object at 0x7fb6bbbd0250>@sqlite:////home/andycarter/vscode_projects/empiric_elt/empiric_meltano/.meltano/meltano.db'
2023-04-19T08:10:43.300424Z [debug    ] Encoding detection: ascii is most likely the one.
2023-04-19T08:10:44.961545Z [debug    ] Found plugin parent            parent=tap-google-analytics plugin=tap-google-analytics source=discovery
2023-04-19T08:10:46.995764Z [debug    ] Found plugin parent            parent=tap-mailchimp plugin=tap-mailchimp source=hub
2023-04-19T08:10:48.227325Z [debug    ] Found plugin parent            parent=tap-tiktok-business plugin=tap-tiktok-business source=hub
2023-04-19T08:10:48.228761Z [debug    ] Found plugin parent            parent=tap-instagram plugin=tap-instagram source=lockfile
2023-04-19T08:10:48.230488Z [debug    ] Found plugin parent            parent=tap-dynamics plugin=tap-dynamics source=lockfile
2023-04-19T08:10:49.611343Z [debug    ] Found plugin parent            parent=tap-facebook-pages plugin=tap-facebook-pages source=hub
2023-04-19T08:10:50.747578Z [debug    ] Found plugin parent            parent=tap-freshdesk plugin=tap-freshdesk source=hub
Installing 1 plugins...
Installing extractor 'tap-mailchimp'...
2023-04-19T08:10:52.171362Z [debug    ] No cached configuration files to remove
2023-04-19T08:10:52.482604Z [debug    ] Removed old virtual environment for 'extractors/tap-mailchimp'
2023-04-19T08:10:52.482802Z [debug    ] Creating virtual environment for 'extractors/tap-mailchimp'
2023-04-19T08:10:53.058410Z [debug    ] Upgrading pip for 'extractors/tap-mailchimp'
2023-04-19T08:10:53.058557Z [debug    ] Upgrading with args '--upgrade pip setuptools==57.5.0 wheel' in existing virtual environment for 'extractors/tap-mailchimp'
2023-04-19T08:10:58.841847Z [debug    ] Installing with args '<git+ssh://git@ssh.dev.azure.com>:v3/EmpiricPowerApps/Sales%20and%20Marketing/tap-mailchimp' into virtual environment for 'extractors/tap-mailchimp'
Extractor 'tap-mailchimp' could not be installed: failed to install plugin 'tap-mailchimp'.
  Running command git clone --filter=blob:none --quiet 'ssh://****@ssh.dev.azure.com:v3/EmpiricPowerApps/Sales%20and%20Marketing/tap-mailchimp' /tmp/pip-req-build-llc0cr5h
  ssh: Could not resolve hostname <http://ssh.dev.azure.com:v3|ssh.dev.azure.com:v3>: Name or service not known
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  error: subprocess-exited-with-error
  
  × git clone --filter=blob:none --quiet 'ssh://****@ssh.dev.azure.com:v3/EmpiricPowerApps/Sales%20and%20Marketing/tap-mailchimp' /tmp/pip-req-build-llc0cr5h did not run successfully.
  │ exit code: 128
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× git clone --filter=blob:none --quiet 'ssh://****@ssh.dev.azure.com:v3/EmpiricPowerApps/Sales%20and%20Marketing/tap-mailchimp' /tmp/pip-req-build-llc0cr5h did not run successfully.
│ exit code: 128
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

And

git clone

```(empiric-elt-py3.10) andycarter@DESKTOP-D6ERPSP:~/vscode_projects/empiric_elt/empiric_meltano$ mkdir tmpdir (empiric-elt-py3.10) andycarter@DESKTOP-D6ERPSP:~/vscode_projects/empiric_elt/empiric_meltano$ cd tmpdir (empiric-elt-py3.10) andycarter@DESKTOP-D6ERPSP:~/vscode_projects/empiric_elt…

Hmm, that message from pip is not very useful. Probably best to file an issue in the pip repo: https://github.com/pypa/pip/issues/new/choose.

@Andy Carter I've had a ton of these myself so I've kinda got good at sussing it out. Instead of

<git+ssh://git@ssh.dev.azure.com>:v3/EmpiricPowerApps/Sales%20and%20Marketing/tap-mailchimp

Try

<git+ssh://git@ssh.dev.azure.com/v3/EmpiricPowerApps/Sales%20and%20Marketing/tap-mailchimp>

A common pattern I've seen fail is

git clone

works great because the

:

is handled, but

pip_urls

don't hand it. Haven't dove into why as I haven't cared enough Thanks for sharing all of that @Andy Carter 😄

It worked! Awesome. I got our admin to remove the whitespace too, not sure if that made any difference but don't care either. It is worth raising an issue to include a reference in the docs?

@Andy Carter yes! Please put it in the docs