https://linen.dev logo
Join Slack
Powered by
I'm getting an error running ELT for the `target-p...
# troubleshooting
l
I'm getting an error running ELT for the 
target-postgres
that I'm not sure how to troubleshoot. I've enabled the debug level of logging but it's not super helpful. The error is 
name=target_postgres level=*ERROR* message=Unable to parse: cmd_type=loader name=target-postgres
but I don't know how to tell what exactly it is "unable to parse"
j
my one week of experience tells me something in your yml is not quite right what messages just above that
l
Great, thanks I'll dig in on that!
j
i have been using the postgres tap, and not the target, but if you want to post what you can...someone might be able to help
l
I just tried removing and re-adding and re-configuring the target and it's still complaining. Would be awesome if i could do 
meltano config target-postgres test
but that doesn't seem to be supported for the postgres target 😕
My config is fairly simple though:
Copy code
loaders:
  - name: target-postgres
    variant: transferwise
    pip_url: pipelinewise-target-postgres
    config:
      host: <my host>
      port: 5432
      user: postgres
      dbname: postgres
      add_metadata_columns: true
and I have the password in the 
.env
file
j
Correct..but you can invoke your target in dryrun i belive.
l
ah thats a good idea, I'll try that out
j
also this its for performance, but.a small file might show you error https://github.com/meltano/meltano/issues/6613#issuecomment-1215074973
i also started with are small data set from a single table...now up to 5 database and 150+ tables
l
ah ok thank you thats good to know about.
I'm still trying to figure out why it's compaining. It almost looks as if its not getting any value for the configs which is causing the JSON parser to fail. 
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Wish I could debug it somehow
j
look up higher in the logs...looks like you might be a ways down...
is it getting an environment correctly? first like should look like [info ] Environment 'prod' is active
l
yeah I'm getting 
2022-12-23T21:06:21.747958Z [info     ] Environment 'dev' is active
and then later it shows: 
2022-12-23T21:06:24.878135Z [info     ] time=2022-12-23 21:06:24 name=target_postgres level=ERROR message=Unable to parse: cmd_type=elb consumer=True name=target-postgres producer=False stdio=stderr string_id=target-postgres
j
then your tap runs?
l
yeah the tap starts and ends quickly. maybe thats the problem is that the tap is returning empty results?
j
that would be a good guess
add the target of target-jsonl
l
okay I'll give that a shot
j
or the first part of the troubleshooting above and look at the results
l
well that was helpful, it looks like i get the same error when targeting jsonl. So must be an issue with my tap instead!
j
what tap?
l
It's a modified version of the Google Analytics tap I've been working on. So the issue must be something that I've done to break it lol. This is my first time in Python so it's a bit of a learning curve at the moment
j
yea, i am still learning python also
l
but when i run the via 
meltano invoke tap-google-analytics
it spits out valid json so i figured it was working
j
and its been years since did anything but sql and bash
yea, dont know if there is a validator for the export format
l
I'm typically develpoing in nodejs. I just need to learn how to get debugging setup w/ python so I can actually troubleshoot better
oh wow... its because I had a 
print
statment in the tap.py and one in the client.py. Makes sense... I guess stdout must be streamed to the extractor. Lesson learned. Thanks for your help @jaye_howell!
j
Glad you figured it out.
Open in Slack
PreviousNext
Powered by