https://linen.dev logo
#singer-tap-development
Title
# singer-tap-development
j

jstark

01/06/2022, 9:43 PM
I’m working on a second iteration on a singer tap that uses the meltano SDK. One of the things I am hoping to do is load json object types as a string into Snowflake. In the target-snowflake README it mentions this limitation of variant or objects types being unavailable in sqlalchemy, and says
Until it is fixed, we are going to store semi-structured data (mainly JSON arrays) as strings and depend on a transformation step to convert those strings to proper snowflake VARIANT or ARRAY types.
The tap fails as I exptected it would if i try to define the object as a string in the object schema with a
Failed validating 'type' in schema
error. I am not sure how to do this conversion from a json object in the tap to a string in the target. Is this a change I would make to the json schema in the tap or somewhere else?