Viet Vu Danh
06/25/2024, 3:29 AMphone_number
in MySQL with type VARCHAR(20)
It's PII so I applied inline mapping hash, the length increased to 32 => it failed on record validations (max length > 20, which is taken from the source length).
• I would like to override the schema so it pass the validation
• I can workaround by mapping to a new field: phone_number_hash
and set null/drop original field phone_number
. However, there are some other cases too, which override schema would solve them.
Therefore I got some questions, it would be great if someone would help me to answer:
1/ Is it possible? Because looks like it's not. I found this issue and it's still open: https://github.com/meltano/meltano/issues/2424
2/ Where in the code that the schema
extra is applied to catalog discovery? I failed to find it in the sdk code.
3/ How can I apply the same logic of catalog discovery to the stream? Overwrite the SCHEMA
message, by modifying tap_base
?Viet Vu Danh
06/25/2024, 3:29 AMEdgar Ramírez (Arch.dev)
06/25/2024, 8:55 AMmeltano.yml
look like?