Want to call out this exciting contribution (WIP) ...
# contributinga
aaronsteers
10/25/2021, 8:53 PMWant to call out this exciting contribution (WIP) from @josh_lloyd: first draft of auto-detecting schema (!193) 路 Merge requests 路 Meltano / Meltano SDK for Singer Taps and Targets 路 GitLab
aaronsteers
10/25/2021, 8:53 PM@visch 馃憜 Would love your eyes and thoughts on this.
v
visch
10/25/2021, 9:03 PMFirst glance this is great, will save a time no question.
Only thing to watch out for is flimsy schemas that may get made, some may also just leave dynamic schema generation on which is ok for some situations but most targets in production you'd rather have a somewhat "static" schema
On my end I'm excited about it, I manually run data through genson, and only 1-2 records at that.
visch
10/25/2021, 9:04 PMAwesome work @josh_lloyd
visch
10/26/2021, 11:44 PMFinally got to grok ~80% of it left a comment. Parent/Child streams is a big question for all of this (I think?)
visch
10/26/2021, 11:45 PMIt's almost like the layer that is doing the schema generation should be moved to ~`get_records` but hopefully I'm just missing something
a
aaronsteers
10/26/2021, 11:52 PMThanks, @visch!
7 Views