What are folks’ thoughts on this? https://www.snowflake.com/blog/choosing-open-wisely/ my hot take is that it’s a lot of words to say they’re not open sourcing certain things ¯\_(ツ)_/¯

I think they conflate a bunch of topics in that post. They have proprietary IP (micropartitioned file formats, for instance) which they could feed into spark or other open standards, but doing so would reduce their competitive edge.

…Abstracting away the details of our internal low-level data representation allows us to continually improve our compression—completely transparently to our users—and deliver other optimizations. The result for customers is lower costs and better performance.”

I’d rather they just come out and say “we compete, at least in part, on the performance of our proprietary data formats” rather than, we are doing this for our customers’ sakes so we can move quickly.

On the topic of integrity of data: Snowflake implemented multi-table transactional consistency since its inception. The query engine coordinates changes and ensures ACID properties are maintained while providing concurrency across workloads and users.

Eventually Delta Lake came in and added ACID for Spark, but Snowflake was first to the game on this IP, and I think they just need to be proud of that accomplishment and make a case why they aren’t open sourcing it.

I agree with your assessment 🙂 I’m still a Snowflake fan, but this was a weird article generally

I just noticed this was the same article in Tristan’s roundup, and I guess he and I have very similar feedback. 😄 Here’s from his comments: To my point of being direct about the business model, Tristan wrote:

The reason you keep code proprietary is so that you have a monopoly on it and can extract rents from that monopoly. That’s fine—it’s called capitalism. (We write a lot of proprietary code too!) But let’s call a spade a spade.

And related to my comment about conflating unrelated points:

There are multiple arguments you could level at the post, though. Here’s the thing that bothers me most: you could achieve all of these architectural benefits while still open sourcing the core technology. And in so doing, you’d create a tremendous asset for all of humanity, something that can never be taken away: a contribution to the knowledge loop.

Honestly, I agree with Tristan’s points 100%. On the flip side, I wonder about the drivers of Snowflake even creating this post in the first place. They must be experiencing some pressure to be writing what appears like a defensive take on them having a closed-source business model.

Spot on AJ - they do sound a bit defensive. They are no more open source than say Microsoft. But I'm not sure why it seems they're embarrassed about it. 🤣

I think it's funny that we're looking at this article today and NASA also issued a release stating the next rover will be open source. Not that they're related obviously.