Hi Everyone - facing the issue while providing multiple variables per pipeline having the similar targets - as suggested in documents or previous slack comments we can pass those within the env object under schedules in meltano.yml but it is not working actually. For the attached metlano.yml it is always dumping the data into output folder instead of dumping into secound_output for another pipeline gitlab-to-jsonl-1.

version: 1
  send_anonymous_usage_stats: false
  plugins:
    extractors:
      - name: tap-gitlab
        variant: meltano
        pip_url: git+<https://gitlab.com/meltano/tap-gitlab.git>
        config:
          projects: meltano/meltano
          start_date: '2020-05-01T00:00:00Z'
    loaders:
      - name: target-jsonl
        variant: andyh1203
        pip_url: target-jsonl
    orchestrators:
      - name: airflow
        pip_url: apache-airflow==1.10.12 --constraint <https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.6.txt>
    files:
      - name: airflow
        pip_url: git+<https://gitlab.com/meltano/files-airflow.git>
  schedules:
    - name: gitlab-to-jsonl
      extractor: tap-gitlab
      loader: target-jsonl
      transform: skip
      interval: '@hourly'
      start_date: 2020-05-01 00:00:00
    - name: gitlab-to-jsonl-1
      extractor: tap-gitlab
      loader: target-jsonl
      transform: skip
      interval: '@hourly'
      start_date: 2020-05-01 00:00:00
      env:
        destination_path: second_output

@ojasvi_harsola Under

env

, you'll want to use environment variables, not setting names! That would look like this:

env:
  TARGET_JSONL_DESTINATION_PATH: second_output

that works, thanks @douwe_maan wondering if we can set these variables run time from UI itself

Pipeline-specific env vars cannot currently be set in the UI, no, but I expect to get to https://gitlab.com/meltano/meltano/-/issues/1394 soon which will let you specify different configuration for each pipeline using profiles.

make sense. thank you.