Implementation of schema object description/comments in the Singer standard and target destination
Hi all! We are in the process of writing many new Meltano mappers, some new taps and enhancing some existing targets. However, we have a use case where we would like to write some additional (human readable) metadata to our target in the form of SQL DDL COMMENT's (such as table description, column description, etc). So far I have not found that this has been implemented yet.
Meltano Singer SDK itself does not seem to have this documented.
https://hub.meltano.com/singer/spec/#schemas
To give you better insight, one of the use cases would be rewriting schema object names (table, column names) - making them shorter due to the length constraints of the destination SQL database. However, it would be good to retain the original object name as a comment in the target.
eg:
table.column: "really-long-table-name.really-long-column-name"
ran through our mapper would be transformed to:
table.column: "really-asd123-name.really-xyz321-name"
and the comments would be then:
table "really-asd123-name": "really-long-table-name"
column "really-xyz321-name": "really-long-column-name"
Any thoughts on this?