Is there a way to do something like this with environment variables? (this doesn't work)

plugins:
  extractors:
  - name: tap-postgres
    variant: transferwise
    pip_url: pipelinewise-tap-postgres
    env:
       PGSSLMODE: verify-ca
       PGSSLKEY: '/home/meltano/ssl/production/product/psql.key'
       PGSSLROOTCERT: '/home/meltano/ssl/production/product/ca.crt'
       PGSSLCERT: '/home/meltano/ssl/production/product/psql.crt'
    config:
      dbname: xx
      host: xx
      port: xx
      ssl: true
      user: xx
      default_replication_method: FULL_TABLE
  loaders:
  - name: target-postgres
    variant: transferwise
    pip_url: pipelinewise-target-postgres
    env:
       PGSSLMODE: verify-ca
       PGSSLKEY: '/home/meltano/ssl/production/analytics/psql.key'
       PGSSLROOTCERT: '/home/meltano/ssl/production/analytics/ca.crt'
       PGSSLCERT: '/home/meltano/ssl/production/analytics/psql.crt'
    config:
      logging_level: DEBUG
      user: xx
      host: xxx
      dbname: xxx
      default_target_schema: xxx
      add_metadata_columns: true
      parallelism: 16

Basically the postgres plugins don't have a config for SSL certs, when the tap and target are the same I can just use the .env file and specify the certs globally but if the tap and target are on different servers, the SSL certs are different

@mark_poole Good question! If we know a plugin supports some env vars, I think we should define settings for them in

discovery.yml

(https://gitlab.com/meltano/meltano/blob/master/src/meltano/core/bundle/discovery.yml#L866) like so:

settings:
- name: ssl_mode
  env: PGSSLMODE
- name: ssl_key
  env: PGSSLKEY
# etc

Any values set for those settings using

meltano config

or some other mechanism will be included in the tap/target's 

config.json

, where they would presumably be ignored, but they'd also be set in the plugin invocation env using the name specified by 

env

. And like any setting, they can be set in 

meltano.yml

, 

.env

, the system database, etc, instead of having to be hard-coded under your suggested

env

property. Would you like to create an issue to add those setting definitions to

discovery.yml

for this tap and target?

@douwe_maan the interesting thing with this is that the plugin doesn't officially support these env vars (they aren't mentioned in the code). I believe the underlying postgres libraries pick them up as default (PGSSLKEY etc)

Ah yeah, makes sense that it's the underlying library. From the user's perspective they're still settings on the tap/target though 🙂