Hi! I'm working to try out Meltano but at my company, we cannot use Python venv due to policy restrictions and are limited to Conda environments. I can create a conda environment to install Meltano itself successfully. However, when attempting to install an extractor (tap-mssql), it appears to be wanting to create a virtual environment for the extractor using venv. Is there an option to do this with Conda instead?

I'm curious: why does such a policy exist?

Hi Brandon!

meltano install

relies on pip and virtualenv, and there’s no way at the moment to use Conda. You could still use Meltano, just without the niceties of `meltano install`: 1. manually install and manage the conda environments for each extractor 2. override the

executable

key in the plugin definition in

meltano.yml

The file would look something like

plugins:
  extractors:
    - name: tap-mssql
      pip_url: null
      executable: path/to/some/conda/env/bin/tap-mssql

@Will Da Silva (Arch) corporate environment, locked down desktops, etc.

I've dealt with many such environments, but I've never heard of

venv

being banned. It's used internally by many Python tools. In my experience, the usual way to lock down Python environments is at the network level by blocking the standard Python package index, and only permitting the use of corporate-controlled Python package indexes. Then there's "bank Python" which is a whole world onto itself...

Agreed. I used to maintain corporate Windows 10 desktops for hundreds of offshore and onshore developers not too long ago and whilst the company (illegally) used the Anaconda distribution, there was no issue at all for me to switch over to the Python.org distribution. All it took was to update the corporate Application Whitelisting rules, which was straight forward because the whitelisting tool in use was the market leader and had all the bells and whistles need for the corporate IT team to implement the needed changes. Lastly, there were always exceptions to the corporate IT policy! (I.e. certain business groups always managed to get the BYOD exceptions extended every year) And the actual policy document should NEVER stipulate the actual software vendor to use. That's not a policy, that's a dictate.

@edgar_ramirez_mondragon do each of the extractors need to have their own conda environments? or is it sufficient to have one conda environment for my entire Meltano project (containing all the plugins)?

You could in theory put all those packages in the same environment, but you're almost sure to get dependency conflicts between them. That's why Meltano uses virtual envs, and why singer.io recommends them.