HI Meltano community, I’d been lurking for about a year while ramping up at a new full time job, related to data.. but focused on deep learning and machine learning. In my spare time I hack of various things.. one of the things that drew my attention to meltano was the DBT integration and the fact that the gitlab team was somehow related. I want to put up my attempt to show you how I’ve started using meltano in my home lab.. and that’s with a tool called Juju. I would bet 10 Swedish Kroner no one knows what it is.. I didn’t either until about 2 years back.. but it’s essentially another orchestration tool… from the team at Canonical. I’ll be brief.. I’ve invested a lot of personal time into Juju.. but it’s given back loads more.. It does vmware (How I started with it), Public clouds, and more importantly private clouds.. I use meltano right now 100% privately to orchestrate extracting data from a source and injecting it into Postgresql.. I’m a big noobie in the data engineering side of things.. but really wanting to see how far I can take meltano to serve myself and my business partner with the data source we use to manage things .. in any event.. I share with you the Juju Charm.. you can think of that either as a helm chart or ansible playbook.. and the starter meltano project.. which you’d likely read quickly and easily. It just takes in a CSV tap.. with a starter CSV and targets a postgres database… the next steps for me after this week are… show how to use this “Meltano Charm” alongside a postgresql charm… the power of juju is when you can “relate” things to eachother.. I see it very akin to cellular biology https://gitlab.com/jrgemcp-public/meltano-project-starter https://gitlab.com/jrgemcp-public/meltano-charm if you want to start trying this yourself in a home lab.. you’d need to install LXD alongside Juju https://jaas.ai/ has more details on that bit It’s a bit late here in Sweden but.. I’ll add a bundle charm next that adds the postfresql bit and continue testing and smoothing out the experience because to me Juju is a VERY incredible tool with a huge learning curve.. but if you write good charms you’ll find it can stand up in VMWare, AWS, GCP, Azure, LXD, etc etc.. with nearly the same YAML that everyone is using with their vendors of choice.

This is awesome! Do you want to come demo it at demo day tomorrow?

let me do some more testing.. but would love to help others get a working meltano ASAP

The next one is in two weeks, no worries. Open offer for whenever you're open to it

Sounds great I don't know anything about Juju! Cool to see

I feel like Canonical is such a great company in some ways.. I loved ubuntu despite a lot of hate for them in the linux community.. but they’ve absolutely NOT shown juju at scale like ansible has been.. it’s baffling

Isn't ansible redhat? I always think Terraform for orchestration haven't heard about juju, but maybe juju is more towards Helm? I have no idea

They used to talk about it at conferences all the time but then they just seemed to stop talking about it