Hello, finally I managed to perform ELT from hubsp...
# troubleshooting
g
Hello, finally I managed to perform ELT from hubspot to sqlite. Thanks for that 🙂 However, I encounter a little problem here: Apparently the data fetched from hubspot was to big or invalid, and it stop the pipeline immediately. Is there anyway to make the pipeline keep going even though it encounter error?
v
Looks like a target-sqllite potential issue. The error of multiple schemas being emitted would actually be a tap-hubspot problem if that's actually occuring. To ignore issues that's a feature that target would have to support. Could try postgres to see if that works better. Or hop in and add the feature to target-sqllite. It's probably worth looking for that duplicated column in the hubspot output. ie
meltano invoke tap-hubspot > hubspotdata
Then taking a look to see if phpbbadc__value is duplicated
I almost wonder if it's happening from the denesting step in hubspot but I'm guessing completely at this point
e
This may have to do with how target-sqlite converts fields names producing two columns with the same name. Maybe you can test with target-jsonl and take a look at those fields with similar names.
v
That makes more sense, target's tend to do the denesting
g
Nice, I will try with jsonl. I also consider another possible solution: modifying catalog.json.
Hi, Jsonl works. I also tried to modify
catalog.json
by running this command:
Copy code
tap-hubspot -c config.json --discover | singer-discover -o catalog.json
and not selecting the problematic properties. Thanks everyone 👍
e
awesome! I'm glad it worked 😄