BuzzCutNorman
03/14/2023, 10:06 PMSTANDARD_KEYS
list and Schema
data class within a tap/loader tap.py
or client.py
? I want to use the JSONSchema keywords contentMediaType
and contentEncoding
that were introduced in Draft 7 but noticed they don't show up in schema messages
until they are added to the STANDARD_KEYS
list and Schema
data class found in singer-sdk\_singerlib\schema.py
. I put some links to more info in a reply to this question.BuzzCutNorman
03/14/2023, 10:19 PMedgar_ramirez_mondragon
03/14/2023, 11:37 PM_extra
field.BuzzCutNorman
03/16/2023, 5:03 PMfrom_dict
method in the Schema
class is the serializer the deserializer is the to_dict
method. Any passed keys not present in STANDARD_KEYS
could be added to _extra
which would be a dictionary. The tap schema message
might look like this "TestColumn1": {"type": ["string", "null"], _extra: {"contentMediaType": "application/xml"}}}
. In the to_sql_type
method on the target side I would look for it with something like this if jsonschema_type.get('_extra', {}).get("contentMediaType") == "application/xml":
.edgar_ramirez_mondragon
03/16/2023, 10:29 PM_extra
to the schema, so you’d see
"TestColumn1": {"type": ["string", "null"], "contentMediaType": "application/xml"}
in the targetBuzzCutNorman
03/16/2023, 10:37 PMto_dict
and from_dict
?BuzzCutNorman
03/16/2023, 10:45 PM"_extra"
to the STANDARD_KEYS
and _extra: dict | None = None
to the Schema
class variables. Then changed to the following in from_dict
and I got what I mentioned earlier "TestColumn1": {"type": ["string", "null"], _extra: {"contentMediaType": "application/xml"}}}
for key in data.keys():
if key in STANDARD_KEYS:
kwargs[key] = data[key]
elif key not in kwargs:
if kwargs.get("_extra"):
kwargs["_extra"][key] = data[key]
else:
kwargs["_extra"] = {}
kwargs["_extra"][key] = data[key]
return cls(**kwargs)