❓ Meltano-UI current state and roadmap Hi Guys, I wanted to know what will happen with the UI? It is deprecated and I totally understand why, focus on CLI. What I don't get though are the implications. Let me explain my point of view for the UI. I configure everything as code. I do not "need" the UI but it is still a great feature to see things (not change or configure) and explain them. Especially with non Data Engineers, it is very usefull for the whole team to get an overview of the Taps, Targets and Pipelines. Additionally, it enables the team to do some things on their own like run a pipeline from the UI if they need fresh data quickly. I think it is a great feature which does not need more feature but still maintenance. As I saw in the release-annoucements, if there were bugs, they have been fixed till now. Will this continue or will you shut down the UI or the maintenance for the UI. Again, I strictly talk about Read-Only in the UI (+ Run Pipeline), nothing else. Thanks for the feedback 😉

Hi @michel_ebner I share your vision of data engineers working with data analysts all on the same Meltano project! In the last office hours, I gave a demo of our Matatika Community Edition - essentially I think this could be a UI for Meltano projects. In our Lab, we give an overview of the taps, targets, and pipelines. Plus simple schedules and starting/stopping of pipelines. DM me if you like, I'd love to hear your thoughts and I'm happy to show you in person. I also have a few videos and demo's specifically for Meltano projects coming soon. Cheers, Aaron

Hi @aaron_phethean sounds interesting. I would definitely want to watch a demo/video with meltano if you have one but I am not sure what the added value would be for us (on a first look) . Without any added value I would not want to introduce yet another tool for my team (at least not while Meltano UI still works).

Yeah, that's fair! We have created a meltano file bundle plugin version of this so it just sits inside your project. Not released yet. One other issue. We took the view that users wouldn't want to setup anything. In our cloud, and the community edition, we collect the logs from the jobs into elastic search. Consequently, it's a bit heavy because you run elastic search and logstash locally. So we've worked out how a simpler UI only container could work. Granted that's still not as simple as 'meltano ui'... I'll drop a short video here.

Hi, @michel_ebner. Agreed the Matatika Community Edition is definitely worth checking out. Aaron and team have created some UI flows and interfaces which are really well designed and helpful, I think. In terms of Meltano's own direction over the next 6-12 months: 1. We are planning to continue our investment in GitHub Codespaces (and similar) which can offer something close to 'instant on' dev environments (locally or in the browser) with no local software required to be installed. (See meltano.com "Try Live Demo" for a sample of this experience as of today.) 2. We are leaning towards a relatively heavy investment in the Meltano VS Code Extension started by @alexander_butler, with ramp up in this area currently slated for Q2/Q3, although this could change and we may start some aspects earlier. Effective immediately, we do welcome community contributions in this area whenever individual community members have need and interest to provide their time and resources in this area. The above 2 investments will stack together in such a way that a 'web based' and also 'IDE-centric' WSYWIG UI should be achievable before end of year. Importantly, these interfaces should address your very well written ask here:

I do not "need" the UI but it is still a great feature to see things (not change or configure) and explain them. Especially with non Data Engineers, it is very usefull for the whole team to get an overview of the Taps, Targets and Pipelines. Additionally, it enables the team to do some things on their own like run a pipeline from the UI if they need fresh data quickly.

Increased investments in the VS Code extension would not imply that users must use VS Code, but this could meet all of your points above, giving folks a more browseable and interactive experience that at the same time will remain connected to the yaml code, and can offer read+write capabilities that will automatically be able to take part in git workflows, even for team members who don't yet feel comfortable with

git

and

meltano

CLI commands.

Hi @aaronsteers Thanks for the long answer and all the information! This helps me and my team a lot to continue to build on the solution! I look forward to see the VS Code extension in action! Keep up the good work 💪

Thanks, @michel_ebner! Really appreciate that. 🙂