Just want to highlight some cool stuff that @alexander_butler is building. We're hoping to get some of this demoed in the coming session(s) but FYI to any contributors interested in something similar, or who might want to test out and contribute back to the work @alexander_butler is doing.

My team is interested in using Meltano with metabase. I'd love to hear more about that!

Yes basically the idea is 2 fold. One, we [can] develop dashboards, charts, locally against a dev instance of Metabase. You then run a meltano utility that extracts the state of the dev instance to its API equivalent (possibly converting to YAML for user modification outside the dev container though some Metabase data structures are interesting since its a clojure backend) Two, we can in CI/CD resolve a mapping of entity ids in production metabase to yaml/json code in source control and PUT request our way through until STATE reflects git. At least in summary

That would be glorious

Indeed 🙂

It's exactly what my team wants. See, we have 2 staging and 4 production environments, with pretty strict requirements that the data can't move from one to the other. So, we want to define a report and have it roll out to 6 environments.

Yes its in early stages but you get the value proposition, especially in a multi tenant isolated environment like that.

Anywhere I can watch to get notifications on progress of this?

The complexity of the solution is almost entirely rooted in how much I want to be able to edit YAML (which implies the need to define useful abstractions in the resource definition yaml that compile to what the API expects) VS yaml being more klutzy with few/no abstractions with the intent that you probably wont want to edit or create them through yaml (though you could) but instead primarily through an accessible dev instance of Metabase on a server or local pc.

Yeah, that's a tough one. I mean, we can already write reports as code if we use d3.js, but I've found that to be extremely time-consuming. Makes sense to use graphical tools to author graphs. We've been looking for "backup/restore" features in BI tools to meet this need. Which lead us to metabase as they have that in one of their paid editions.

I think we will get it "working" with the latter (klutzy, but still editable and creatable yaml validated with pydantic), then ideally move towards the convenient abstractions once we feel good and temperature check/determine the work is worth it

Hey @alexander_butler how is the meltano-metabase integration work going?

Not sure on @alexander_butler work, but @pat_nadolny is pushing the plugin definition onto the hub right now, so give it another few days and you should be able to pull it. It already mounts the data from the meltano proj, so presumably this will indeed version your metabase reports & dashboards. https://github.com/meltano/hub/pull/865

Oh that's exciting!

Thanks @Sven Balnojan - that PR does the easy part, it just adds metabase as a discoverable plugin but doesnt add any extra functionality to it like exporting reports/dashboards yet. Now you're able to install and spin up metabase easily (

meltano add utility metabase

then

meltano invoke --containers metabase:ui

). We also then have a path forward for where to put the EDK wrapper functionality once its written. I could see that extra functionality becoming additional commands so we spin up the UI like I described above (maybe in the background) then run something like

meltano invoke meltabase:export

to pull out all the reports as yaml into your repo.

@pat_nadolny Good to know! I just saw you're already mounting a local directory, so we need some export/import functionality with a decent workflow and then what @dylan_just is looking for is possible right (versioning the complete ELT-Reporting pipeline)?

For now, we're going to just play with the Metabase Pro Serialization feature and work it into either our meltano project, or the project where we have our Metabase deployment scripts.