Howdy, I'm struggling with extracting data from My...
# troubleshooting
m
Howdy, I'm struggling with extracting data from MySQL's column of type "json" using tap-mysql. Has anyone ever done it successfully? my config is very basic, just pointing at mysql and selecting a table with metadata column of json type. I get the following result regardless of target (jsonl, csv, snowflake):
Copy code
2024-01-31T13:22:52.019876Z [info     ] jsonschema.exceptions.ValidationError: '{"enableAttributes": "true", "defaultAttributes": "true", "mobileOnlineViewerUrl": "<http://localhost/>"}' is not of type 'null', 'object' cmd_type=elb consumer=True name=target-csv producer=False stdio=stderr string_id=target-csv
2024-01-31T13:22:52.019876Z [info     ]                                cmd_type=elb consumer=True name=target-csv producer=False stdio=stderr string_id=target-csv
2024-01-31T13:22:52.019876Z [info     ] Failed validating 'type' in schema['properties']['metadata']: cmd_type=elb consumer=True name=target-csv producer=False stdio=stderr string_id=target-csv     
2024-01-31T13:22:52.019876Z [info     ]     {'inclusion': 'available', 'type': ['null', 'object']} cmd_type=elb consumer=True name=target-csv producer=False stdio=stderr string_id=target-csv        
2024-01-31T13:22:52.019876Z [info     ]                                cmd_type=elb consumer=True name=target-csv producer=False stdio=stderr string_id=target-csv
2024-01-31T13:22:52.019876Z [info     ] On instance['metadata']:       cmd_type=elb consumer=True name=target-csv producer=False stdio=stderr string_id=target-csv
2024-01-31T13:22:52.019876Z [info     ]     ('{"enableAttributes": "true", "defaultAttributes": "true", ' cmd_type=elb consumer=True name=target-csv producer=False stdio=stderr string_id=target-csv 
2024-01-31T13:22:52.028360Z [info     ]      '"mobileOnlineViewerUrl": "<http://localhost/>"}') cmd_type=elb consumer=True name=target-csv producer=False stdio=stderr string_id=target-csv
So, the short story is that I found a code snippet to fix it and I forked tap-mysql and applied the change here: https://github.com/transferwise/pipelinewise-tap-mysql/compare/master...BamboOSZ:pipelinewise-tap-mysql:master It's send to snowflake as varchar so needs parsing (PARSE_JSON), but all in all it works fine!
the change to plugin: