Hello everybody! Someone know what it's this error...
# troubleshooting
p
Hello everybody! Someone know what it's this error? I use a pipeline using target-bigquery and honestly I didn't found some clear to what it's or what can I do.
m
does it work with target-jsonl?
p
This is a pipeline that use a custom tap-intacct and target-bigquery, if I use with target-csv it works. The pipeline works but with some files I get this error
m
Tough to see what the error is given the short message on the screenshot. If it works with target-csv and target-jsonl then it’s not your extractor. Are you running
meltano el tap-intacct target-bigquery
? If you run
meltano el tap-intacct target-jsonl
it works without error right?
p
I running meltano run tap-inntacct target-bigquery, I don't attempt yet with meltano el. Is better if I use el?
m
I think either is fine. Seems like
run
more catered for customized pipelines beyond just extract and load but if it’s a simple extract and load I think
el
is preferred (I don’t think it makes much of a difference though). I noticed something extremely strange when building my recent tap and couldn’t find an answer ANYWHERE on any thread in slack or in any documentation in singer or meltano docs. In my situation, I was getting
Loader Failed
but my extractor ran perfectly fine with
meltano invoke
. After about a full week of debugging I found out one of the 3rd party libraries I was using called a
print
function in their code. This causing the entire loading process to break (still don’t know why). There must be something in either meltano or singer code somewhere where it specifies that you cannot use
print
statements. They must be used with
self.logger.<info>
. Not sure if that is your problem, but it was a huge headache for me.
but i’ll say if it’s working with target-csv then that shouldn’t the issue.. Must be specific to that target-biguqery configuration. Maybe try a different variant of target-bigquery or check out the github/docs for that target
p
Ok I'll review the documents and attempt with the other variant if the problem persist. Thanks for your help