Hi folks after upgrading to latest meltano I m seeing this a Meltano #troubleshooting

Hi folks, after upgrading to latest meltano, I’m s...

chris_kings-lynne

04/29/2021, 9:35 AM

Hi folks, after upgrading to latest meltano, I’m seeing this at the end of pretty much every salesforce tap stream completion:

chris_kings-lynne

04/29/2021, 9:35 AM

It’s rated as an error, but doesn’t really seem to affect anything. Is it a bug?

visch

04/29/2021, 2:59 PM

asyncio errors can be fun, it is a bug. Could you run the job in debug mode, and set an env variable of

PYTHONASYNCIODEBUG=1

https://docs.python.org/3/library/asyncio-dev.html re That would be helpful I think to whoever is trying to debug this. The hard thing is trying to replicate the error as something is stopping the event loop, who knows what. I'm curious what the Meltano dev team thinks about this one

douwe_maan

04/29/2021, 3:21 PM

@chris_kings-lynne This started happening when you upgraded from Meltano v1.71.0 to v1.72.0? Does it disappear again if you go back to the previous version? v1.72.0 does include some changes related to asyncio (https://gitlab.com/meltano/meltano/-/merge_requests/2105), but the fact that the lines are prefixed with

tap-salesforce |

rather than

meltano |

suggests that the error is actually coming from tap-salesforce, not Meltano. tap-salesforce also uses asyncio (https://gitlab.com/meltano/tap-salesforce/-/blob/master/tap_salesforce/__init__.py#L374-375), so it could definitely have some bug there, but I don't understand why it would only start showing after the upgrade, since Meltano's asyncio changes shouldn't affect asyncio inside the tap.

douwe_maan

04/29/2021, 3:22 PM

Like Derek mentioned, try running again with

PYTHONASYNCIODEBUG=1

to see if we get some more useful logs from Meltano or the tap itself

chris_kings-lynne

05/02/2021, 3:22 AM

Mmm I think I figured it out. It was one of the streams failing because I was using bulk api and it cannot fetch geolocation types. The failure in that thread was polluting all sorts of downstream ones perhaps?

josh_lloyd

07/29/2021, 9:40 PM

sorry to dig this old thread up, but @chris_kings-lynne I was wondering if you remember how you fixed this. I’m getting the same error. The root error seems to be:

Copy code

tap-salesforce | CRITICAL Error syncing Opportunity: InvalidBatch : Failed to process query: FUNCTIONALITY_NOT_ENABLED: Selecting compound data not supported in Bulk Query

I tried to deselect the potentially offending fields like this

Copy code

"!Opportunity.LID__My_Geolocation__c.longitude"

but because it’s “automatically” selected, this doesn’t work. Any tips? Do I have to switch the

api_type

chris_kings-lynne

07/30/2021, 2:17 AM

Copy code

- name: tap-salesforce
    variant: meltano
    pip_url: git+<https://gitlab.com/meltano/tap-salesforce.git@v1.4.27>
    config:
      api_type: BULK
    select:
    - Account.*
    - AccountContactRelation.*
    - Attachment.*
    - '!Attachment.Body'
    - Campaign.*
    - CampaignInfluence.*
    - CampaignMember.*
    - CampaignMemberStatus.*
    - Contact.*
    - EmailMessage.*
    - '!EmailMessage.*Body'
    - Event.*
    - Lead.*
    - LeadStatus.*
    - Locality__c.*
    - '!Locality__c.Geolocation__c'
    - Opportunity*.*
    - Pricebook*.*
    - Product2.*
    - RecordType.*
    - Task.*
    - User.*
    - '!*Feed.*'
    - '!*History.*'
    - '!*Share.*'

chris_kings-lynne

07/30/2021, 2:21 AM

Enjoy

chris_kings-lynne

07/30/2021, 2:21 AM

the bottom 3 get rid of all the feed, history and share tables. you can keep them if you want

Open in Slack

Previous Next