Hi, I'm working with tap-salesforce and trying to control the concurrency. so I passed the max_workers parameter as environment variable, but the problem is that it is being parsed as string instead of int and I am getting the following:

TypeError: '<=' not supported between instances of 'str' and 'int'

Can you help please?

@or_barda Good catch! This likely has to do with the fact that this setting is not currently defined in the plugin definition at https://gitlab.com/meltano/meltano/blob/master/src/meltano/core/bundle/discovery.yml#L578, so Meltano doesn't know what type it's supposed to be. Can you please share the

tap-salesforce

plugin definition from

meltano.yml

so that I can suggest an appropriate workaround?

Hi Douwe, thanks for the answer. Right now I'm hard coding the the number but this is the yaml file:

extractors:
  - name: tap-salesforce
    pip_url: git+<https://gitlab.com/meltano/tap-salesforce.git>
    namespace: tap_salesforce
    executable: tap-salesforce
    capabilities:
    - catalog
    - discover
    - state
    select:
    - Entitlement.*
    - Pricebook2.*
    - CaseTeamTemplate.*
    - pse__Proj__c.*
    - PSA_Case__c.*
    - OpportunitySplitType.*
    - SBQQ__DiscountSchedule__c.*
    - Merchant_ID__c.*
    - Engagement_Score__c.*
    - CaseTeamTemplateMember.*
    config:
      start_date: '1900-01-01T00:00:00Z'
      username: 
      password: ${SF_PASSWORD}
      security_token: ${SF_SECURITY_TOKEN}
      api_type: BULK
      select_fields_by_default: true
      max_workers: $SF_MAX_WORKERS

@or_barda If a setting is not explicitly defined in the plugin definition and and only has a value in

meltano.yml

, Meltano will determine the type by looking at that value, which is currently a string:

"$SF_MAX_WORKERS"

. If you set it to an integer instead (e.g.

max_workers: 8

), you can still override it using the env var that shows up in

meltano config tap-salesforce list

(

TAP_SALESFORCE_MAX_WORKERS

), but now Meltano will know that the value should be cast to an integer before being passed to the plugin. Now that the setting has been added to the discoverable definition with an explicit

kind: integer

(https://gitlab.com/meltano/meltano/-/merge_requests/1907), you can also try removing the

namespace

,

executables

and

capabilities

properties so that Meltano will fall back on the discoverable definition.