I feel as though there ought to be a way to accomplish this: Can i extract a sql expression instead of the column, or extract binary columns. Context: I have a table in mysql with a column (called extra_props) of type mediumblob where some json structured data is stored. I would like to extract the json, but mediumblob is not supported as a column type, and i get the following error message:

... level=WARNING message=Columns {'extra_props'} were selected but are not supported. Skipping them. ..

The tap configuration mentions this field like this:

select:
  ...
  - my_table.extra_props

if we can get meltano to execute

select ..., convert('extra_props' using utf8mb4) as extra_props from mytable

, i feel like we can avoid the unsupported type behavior. i was also thinking of forking the singer.io/tap-mysql and adding support for binary data, where the binary data types get b64 encoded, but I feel like the first option is more sane.

@haleemur_ali a couple thoughts here: 1. This isn't really a Meltano problem - it's a tap-mysql problem 😉 2. You can accomplish this using a view, but only if you can use full_table or incremental replication mode 3. Extending the tap to support blob columns should work. I think. 😝

Thanks, I've chosen to go with option 3. the transferwise variant has partial support for binary data already (_they hex encode the binary column) here, and i believe to get blob & mediumblob working, i have to extend the set

BINARY_TYPES

with a couple more entries here_ this will result in the binary data ending up as a hex encoded string on the target.