Hey, just started building internal analytics for ...
# troubleshooting
j
Hey, just started building internal analytics for GoodData, my first real-world solution with Meltano. Tapping Salesforce(horrible service).
Copy code
2023-03-01T17:14:11.926648Z [info     ] INFO METRIC: {"type": "counter", "metric": "record_count", "value": 55618, "tags": {"endpoint": "Task"}} cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.807070Z [info     ] INFO METRIC: {"type": "counter", "metric": "record_count", "value": 56799, "tags": {"endpoint": "Task"}} cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.807608Z [info     ] ERROR exception calling callback for <Future at 0x7f2ce89b1e10 state=finished returned NoneType> cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.807866Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.808110Z [info     ]   File "/usr/lib/python3.10/concurrent/futures/_base.py", line 342, in _invoke_callbacks cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.808337Z [info     ]     callback(self)             cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.808556Z [info     ]   File "/usr/lib/python3.10/asyncio/futures.py", line 399, in _call_set_state cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.808765Z [info     ]     dest_loop.call_soon_threadsafe(_set_state, destination, source) cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.808976Z [info     ]   File "/usr/lib/python3.10/asyncio/base_events.py", line 795, in call_soon_threadsafe cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.809185Z [info     ]     self._check_closed()       cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.809395Z [info     ]   File "/usr/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.809615Z [info     ]     raise RuntimeError('Event loop is closed') cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
2023-03-01T17:14:50.809825Z [info     ] RuntimeError: Event loop is closed cmd_type=elb consumer=False name=tap-salesforce producer=True stdio=stderr string_id=tap-salesforce
Does anyone know what could cause this? When I limit the list of entities e.g. to Account, Lead, OpportunityHIstory, and a few others, it works well. Looks like the suspect here is the Task entity?
Hmm, looks like this is the root cause:
Copy code
CRITICAL Error syncing Opportunity: InvalidBatch : Failed to process query: FUNCTIONALITY_NOT_ENABLED: Selecting compound data not supported in Bulk Query
I am pretty sure that OpportunityHistory does not suffer from this issue. I tried to switch from BULK to REST, but it is veeery slow
OK, reaching API quota, cannot use REST way. Anyone can help to make the Opportunity entity in BULK mode work?
Ha, it is about particular (compound) keys. Have to specify exact set of fields, from which none can be compound.
a
Which tap-salesforce are you using specifically? Mine which is just the meltano labs fork basically, though I did stab at an SDK port, will do batch extracts fine and ignore what is unsupported
j
I use
pip_url: git+<https://github.com/meltanolabs/tap-salesforce.git@v1.5.0>
. I specify the following fields for the Opportunity entity:
Copy code
- Opportunity.Id
      - Opportunity.Name
      - Opportunity.StageName
      - Opportunity.Booking_Type__c
      - Opportunity.CreatedDate
      - Opportunity.CloseDate
      - Opportunity.OwnerId
      - Opportunity.ARR_Net_New__c
      - Opportunity.Year_1_Platform_Estimate__c
      - Opportunity.Sales_Channel__c
It still fails with the error. Any advice how to overcome this issue will be more than welcome 😉
Please, anyone, how could I fix this?
a
I dont have that issue, but maybe try pinning to
main
(ie drop the
@1.5.0
) as there were some commits including using a newer SF API version since that release, so its stale