Hi everyone I m experiencing an issue with `meltano invoke t Meltano #troubleshooting

Hi everyone, I'm experiencing an issue with `melta...

courtney_wright

10/27/2021, 4:51 AM

Hi everyone, I'm experiencing an issue with

meltano invoke tap-mytap

which I'm not sure how to resolve. I'm not getting any records in the output and the cli command output shows that the record count is 0

INFO METRIC: {'type': 'counter', 'metric': 'record_count', 'value': 0, 'tags': {'stream': 'company_call_recordings'}}

. My out.jsonl shows a "type": "SCHEMA" object and a "type": "STATE" object. The api requests are all succeeding. If I set a breakpoint in

parse_response()

, I can see the response is as I expect and has valid data in it. The

records_jsonpath

is correct (tested with the reponse on jsonpath.com). I've tried simplifying my schema to just extract the id from each object in case that was the problem but it didn't make a difference. I haven't set up any select filters. I'm not sure what else I can check here to figure out what's missing. I'm running meltano via docker with the image meltano/meltano:latest-python3.9 and it's version 1.84.0.

edgar_ramirez_mondragon

10/27/2021, 3:12 PM

Hi @courtney_wright. Are you developing your tap with poetry? In that case can you try invoking "manually" like

poetry run my-tap --config config.json

just to rule out Meltano is somehow deselecting the streams?

courtney_wright

10/27/2021, 11:28 PM

I found that I wasn't setting the log level correctly, once I did I found the actual issue was

Copy code

File "/usr/local/lib/python3.6/site-packages/meltano/core/plugin/singer/tap.py", line 42, in _stream_redirect
    file_like_obj.write(data.decode("ascii") if write_str else data)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 2: ordinal not in range(128)

so it looks like I've got to modify it to use utf-8 encoding instead

courtney_wright

10/28/2021, 12:39 AM

Hm turned out this line was the problem (in

parse_response()

)

Copy code

yield from extract_jsonpath(self.records_jsonpath, input=response.json)

I replaced it with this instead (just return a list of dictionaries) and it works

Copy code

r = response.json()
recordings = []
  if "_embedded" in r:
    recordings.extend(r["_embedded"]["recordings"])
return recordings

Open in Slack

Previous Next