Can someone quickly outline why Meltano is a bette...
# random
e
Can someone quickly outline why Meltano is a better choice for enterprise than PipelineWise, and what things should be considered? Seems like a lot of things are still getting worked out or are buggy. At least for people somewhat unexperienced with python and etl's. I found a lot of accounts of people using the first for a long time in production, but not so much on Meltano.
a
Hi Espen, I'd love to hear a bit more about the bugs you've experienced. We have a few enterprises using Meltano but we're still relatively new which is why you may not see a lot about longevity yet
p
@espen_espelund good question - here my take! A major difference is that the vision of Meltano is larger than just EL, that’s where the DataOps OS naming comes from. Meltano’s goal is to make installing, configuring, and running data tools together easier. At this Data OS layer were able to bring all the tools in the stack together to work in harmony while also giving teams the tools to implement good software engineering practices (isolated environments, CI, testing, code reviews, etc.) for their data stack. Check out https://gitlab.com/meltano/squared that’s our own data platform built using Meltano with plugins like Singer/dbt/Airflow/Superset/Great Expectations/Permifrost/aws cli/ etc. And before I started working at Meltano (the company) I personally ran Meltano (the software) at a large enterprise with success! We implemented it years ago so we were only using it for EL (and reverse ETL) though.
Specifically for EL - Meltano has features to enable better DataOps like https://docs.meltano.com/concepts/environments and https://docs.meltano.com/reference/command-line-interface#test and it supports the 300+ Singer taps/targets https://hub.meltano.com/singer/ along with a python SDK for building new ones. Meltano supports the "T" in ELT also with dbt as a plugin.