https://meltano.slack.com/archives/C01TCRBBJD7/p1670314985985749?thread_ts=1670312879.839659&cid=C01TCRBBJD7 I've seen variations of this in tap-postgres with Objects and Arrays. I'm working on a fix now, but the key thing here is the fix we currently have for converting to strings when all else fails doesn't work unless we also convert the values for that corresponding schema to a string as well

On that topic, I was doing a bit of light reading on

TypeEngine.as_generic()

which is intended for transferring schemas between different database vendors. https://docs.sqlalchemy.org/en/20/core/reflection.html#metadata-reflection-dbagnostic-types

There is a use case for reflection which is that a given Table is to be transferred to a different vendor database. To suit this use case, there is a technique by which these vendor-specific datatypes can be converted on the fly to be instance of SQLAlchemy backend-agnostic datatypes, for the examples above types such as Integer, Interval and Enum. This may be achieved by intercepting the column reflection using the DDLEvents.column_reflect() event in conjunction with the TypeEngine.as_generic() method.

Sounds nice we could probably take advantage of that, I don't think it'd solve this issue but looks neat 😄

Yeah. I was initially reading up on it due to the "MONEY" / "SMALLMONEY" topic in the original thread. I think there may be more weird data types out there where the SDK may currently fail to map to JSON schema types in a way that works as a user would expect it to work

for sure, what should happen is the fall back of string should work and convert the data type to a string so instead of failing it atleast passes the data (even if it's the wrong format)