howdy wave the `target bigquery` throws and `OBJECT is not a Meltano #troubleshooting

howdy :wave: the `target-bigquery` throws and `OB...

finn_frotscher

09/30/2021, 8:12 AM

howdy 👋 the

target-bigquery

throws and

OBJECT is not a valid value

error when data from

hubspot-tap

is piped into it.

Copy code

cat hubspotdata | meltano --log-level=debug invoke target-bigquery

how do I debug somehting like this? any idea what causes this?

Copy code

INFO Appending to t_test-contact_lists_0ca392f6d2a24f889f52fcad32ca20c1
INFO loading t_test-contact_lists_0ca392f6d2a24f889f52fcad32ca20c1 to BigQuery
ERROR failed to load table t_test-contact_lists_0ca392f6d2a24f889f52fcad32ca20c1 from file: 400 POST <https://bigquery.googleapis.com/upload/bigquery/v2/projects/warehouse-310706/jobs?uploadType=resumable>: Invalid value for type: OBJECT is not a valid value
CRITICAL 400 POST <https://bigquery.googleapis.com/upload/bigquery/v2/projects/warehouse-310706/jobs?uploadType=resumable>: Invalid value for type: OBJECT is not a valid value

visch

09/30/2021, 11:45 AM

Submitted an issue for this as it relates to https://meltano.slack.com/archives/C01PKLU5D1R/p1632247195183600 https://gitlab.com/meltano/meltano/-/issues/2958

visch

09/30/2021, 11:47 AM

From here I'd try to narrow down your output file, it looks like it's contact-list specefic, so maybe update your

meltano.yml

to just pull contact-list information. Verify you can production the issue. Then If you still don't see the issue, I'd slap some print ln's in target-bigquery around wherever that error is happening. Since there's not a stack trace it makes this a little harder

edgar_ramirez_mondragon

09/30/2021, 7:18 PM

Hey @finn_frotscher . So from looking at the contact_lists schema, it seems like there are 2 potentials fields causing the issue for bigquery. Either

filters[*][*]

metaData

. So within

hubspotdata

you might wanna look for strange values of those that are not actual JSON objects.

finn_frotscher

10/01/2021, 7:45 AM

thank you. so far no progress 😞 i will keep you posted

finn_frotscher

10/01/2021, 8:42 AM

its not the contacts. its

contact_lists

. the contacts sync fine, but are in some kind of nested structure.

Copy code

select:
      # - contacts.*
      - contact_lists.*

finn_frotscher

10/01/2021, 9:03 AM

since i am getting the data i need for now, i wount investiage further. should i open a ticket ?

edgar_ramirez_mondragon

10/01/2021, 4:12 PM

If you're willing to, yes please. The issue tracker for tap-hubspot is https://github.com/singer-io/tap-hubspot/issues

Open in Slack

Previous Next