Hey folks, with target-MySQL the column names are being changes from uppercase first letter to all lower case
This is affecting the recognition of my primary keys. Is this behavior default?
Ex. I have a primary key "Title" and I have my JSON schema as such, but I get a key error. Do I need to add a mapping so that it is "title" ?
edward_ryan
04/18/2024, 6:03 AM
Another example, the stream from API is called "HashedEmailAddress" and that's how we have the column named in our inline schema. However, it appears in the table as "hashed_email_address" and results in a primary key error
target-mysql seems to be changing the camel case to snake case automatically which is fine but affecting primary key declaration
edward_ryan
04/18/2024, 6:12 AM
We have a workaround using post_process that resolved the error
But I don't think this is best practice
Copy code
def post_process(self, row: dict, context: dict | None = None) -> dict | None:
"""
We can add created and updated time columns from field column
"""
try:
row["title"] = str(row["Title"])
except:
pass
return super().post_process(row, context)
v
visch
04/18/2024, 5:14 PM
I didn't write target-mysql so whoever wrote that is what is causing the default behavior you're seeing.
Sounds like a bug to me with target-mysql 🤷
visch
04/18/2024, 5:16 PM
You could do a schema override with your tap and manually add
hashed_email_address
just to get you by, but I wouldn't dive too far down that rabbit hole to get it working I"d just look at the target and try to fix whatever is going on.
visch
04/18/2024, 5:16 PM
Sharing your logs would be helpful too
e
edward_ryan
04/18/2024, 5:17 PM
the automatic conversion of camel case to snake is a bug in the target?
ok, I'll get the logs
to confirm, this post_process approach isn't a good idea?
question: how do we do the schema override, is that a mapping to align the streams?
thank you!!