Hi folks! Is there a variants guide? Reasons to us...
# plugins-general
d
Hi folks! Is there a variants guide? Reasons to use one vs. another, or a description of why a variant exists at all? For example, I'm looking at the target-bigquery options and there are four. Short of installing and trying all four, what's the best way to choose between them?
v
My quick version is look at how many projects are running the variant, there's numbers on the hub 🤷
Past that I install them and try it
Hardest part is everyone wants something slightly different, so the full answer to everyone's question is to "read the code" or run it
Past that what would you want to see Danylo?
p
For target-bigquery specifically we were considering making https://hub.meltano.com/loaders/target-bigquery--z3z1ma/ or https://hub.meltano.com/loaders/target-bigquery--youcruit/ the default because theyre SDK based and have more development going into them
Also Alex shared some performance milestones in https://meltano.slack.com/archives/C04632W0HT2/p1678820618397579
d
@visch that's what I've been doing so far - project count, github check. What's missing for me is a couple of sentences from the developer of (a) why a variant exists and (b) support plan. I know all of these are free to use, so there's no real support commitment but it would be nice to know if the dev is still maintaining it.
a
Bumping this so it isn't buried, We have more contributors, more users (in theory, with 12K executions vs 50 on the hub fwiw), probably more exposure as a result, benchmarks that have hit >1MM records per minute which is 95% faster than the benchmark in the blogpost without even using BATCH; but benchmarks aren't worth much unless it is on the exact same environment/network/VM/etc so its kind of moot. There are 20 different ways to load your data depending on your preferences and workload. Storage write support is quite compelling. A pretty extensive end 2 end integration test to a live BQ in our CI with very precise assertions. (a now pinned SDK version + dependabot 😉). Many months of production usage at Harness. So the question is: What compelling and interesting feature is not present in my variant that is present in the other? I'm open to whatever really, feedback or otherwise. Also open in general to onboard any good ideas from the other target into mine and/or to bring on more motivated maintainers. I think our variant is due a metadata refresh on the hub soon. But otherwise seems solid.