What interest would there be in a Go SDK for Melta...
# singer-tap-development
f
What interest would there be in a Go SDK for Meltano/Singer taps and targets?
a
Hi, @fred_reimer - we're moving so quickly with features and bug fixes on the Python side, and there are just a lot more Python coders out there (esp. in the data community) - I think it could be tricky to additionally have a go-based SDK additionally. That said, I absolutely would encourage this if you are passionate about it. I do want to ask though - are you mostly interested because you love Golang? or is part of your motivation a drive for the most-optimal performance? If performance is primarily what you are after, I wonder if a hybrid approach would be worth pursuing? This is a bit stale, but the title of this article seems to suggest a Python SDK could take advantage of Go backends when raw performance is what matters most. I've heard of similar implementations combining Python with Rust. I'm very interested in your thoughts about this, and to understand your specific interests and values. If raw performance is all that matters, I could imagine a Python SDK "frame" invoking compiled libraries for low-level record-level operations, while still being able to take advantage of the Python SDK's features out of box.
I don't have tons of experience with Go, but I do like the language a lot. I learned Go for some Terraform contribution work a year or so ago - it's a cool language with some great design decisions, IMHO.
f
I ask for several reasons: 1. The necessity of Python env's per plugin to avoid module conflicts - go is a single binary 2. Speed as far as compiled/interpreted code. Don't want to over-emphasize this, as it is not that bad, but there is a difference. 3. The actual code for taps and targets is rather straight-forward, and don't necessarily benefit greatly from a dynamic typed language like Python as far as I can tell. Go has ways of dealing with this if necessary though. 4. I like go! 5. Threads, or multi-processing in Python is "difficult" at best. Go routines would alleviate further performance issues, if there are any for a particular use-case. That said, it would be a personal project, so development would probably be intermittent as I get time.