Hey everyone, happy monday! I was wondering if anyone had any best practices on developing private taps. I'm currently creating a tap that would only be used for internal use, and I'm wondering if it would be best practice to keep it private or make it public (no one would have the token anyway, so it shouldn't be too much of a security issue). I was looking into this thread to work with private taps, so I'm also exploring on how to work with that in production as well. Thanks!

@Stéphane Burwash I don’t think there’s anything special you need to do AFAIK. You just need to enable auth via the pip_url. We have some docs on that https://docs.meltano.com/guide/plugin-management#using-a-custom-fork-of-a-plugin

Awesome thanks! I think the logic I was missing the the

.netrc

file. It should automatically be captured by my meltano project?

I would assume yes but I’ll lean on our engineers to confirm - cc @Will Da Silva (Arch) @edgar_ramirez_mondragon

Yes,

pip

which meltano uses under the hood should automatically use the

.netrc

file when installing from a git url. Another option I used in the past file is to publish private taps to an s3pypi repository and use

PIP_EXTRA_INDEX_URL

pointing to it.

Does the

.netrc

file work with ssh, or does it only work with ssh?

one of our brilliant engineers would probably be best to help with that! @pat_nadolny or @Will Da Silva (Arch)? 😄

I'm not an expert on this but I think the main thing is making sure you remove your SSH key once the install is done so they arent built into your image. Ideally youd want to add the keys, install your plugins, then remove them. I think this is a common thing for docker, this stack overflow answer seems relevant

Not ideal when you're doing development since we would have to rebuild the image everytime, but definitely better than making the repository public. Thansk! @pat_nadolny 😄 Would be very cool in the future to have a feature to specify a SSH key directly in your meltano env

For development you should be able to use a volume to mount your local SSH configs into the container so it has the same access that your machine has

Awesome thanks! I'll try that out 😄