https://linen.dev logo
Join Slack
Powered by
Hello I've been working on a custom tap to take da...
# troubleshooting
c
Hello I've been working on a custom tap to take data from an API. I'm trying to make it run now but I'm having the following log: 
File "/project/.meltano/extractors/tap-whise/venv/lib/python3.6/site-packages/singer/schema.py", line 98, in from_dict
properties = data.get('properties')
AttributeError: 'list' object has no attribute 'get'
when running my tap. Can you help me out?
l
Are you giving it a properties thing?
Untitled
Untitled
Should look something like this
I'm thinking you might be giving it a list which fails because it's expecting a dict
c
I'm not giving the tap any properties: I did not add it as a spec in my tap
Isn't the discover capability able to create that list?
l
I guess it should be able to
I guess you might be doing something wrong with the custom tap but it's difficult to tell what, it seems that the schema that it generates is bad for some reason
c
This is my current schemas for instance : 
representatives': {'$schema': '<http://json-schema.org/draft-04/schema#>', 'type': 'object', 'properties': {'Id': {'type': 'string'}, 'DirectLine': {'type': 'string'}, 'Email': {'type': 'string'}, 'FirstName': {'type': 'string'}, 'Mobile': {'type': 'string'}, 'Name': {'type': 'string'}, 'OfficeId': {'type': 'string'}, 'PictureUrl': {'type': 'string'}, 'UpdateDateTime': {'type': 'string'}, 'Commission': {'type': 'string'}, 'CommissionUnit': {'type': 'string'}}}}
maybe it's not formatted properly?
Or it's seen as a string instead of a dict?
It's a dict : 
<class 'dict'>
One of the stream must not have a propeties property, maybe
l
Untitled
Untitled
this is what my schema looks like
(simplified)
should be possible to write a similar one for yours
and see what it does when you give meltano/singer that
c
Indeed, mine do not look like that. I'll check that option further now. I'll keep you posted! Thanks Lasse 🙂
Do you have any tool that helped you creating those schemas?
@lasse_vang_gravesen Thanks for your help! Just went on the meltano documentation about the schemas. And modified the underlying schemas. I no longer have this error 🙂
l
I had mine autodiscover the schema
Nice, good!
a
@charley_guillaume - glad you got this worked out! We also have helper classes to make it easier to construct schemas. If you ever want to revisit how you are managing those, just let us know.
c
Ho! I'd definitely have a look at those! Can you point me to those helper classes? Thanks @aaronsteers
a
In the links at the top of Code Samples — Meltano SDK 0.2.0 documentation, you can drill into most any of those existing implementations. But specifically, the Power BI example has an extremely detailed schema which is defined entirely via helper classes: https://github.com/dataops-tk/tap-powerbi-metadata/blob/main/tap_powerbi_metadata/streams.py#L119 Note at the very bottom of that declaration, theres a 
.to_dict()
call, which converts the entire structure back into a standard JSON Schema dictionary structure.
I personally prefer declaring JSON Schema with the helpers because then I can rely on type hinting, intellisense, etc. and I ac worry a lot less about malformed json or random typos.
The SDK supports either of these two options: 
Stream.schema
(expected to be a python dictionary) and 
Stream.schema_filepath
(expected to be a path to a json text file).
7 Views
Open in Slack
PreviousNext
Powered by