Dagster integration was mentioned in the livestrea...
# contributinga
aaronsteers
06/09/2022, 5:02 PMDagster integration was mentioned in the livestream demo day today! New 🧵 to talk about steps to get on the Hub! melty bouncy
aaronsteers
06/09/2022, 5:03 PMFrom @josh_lloyd https://meltano.slack.com/archives/C01NPCGEL48/p1634568486052000
aaronsteers
06/09/2022, 5:04 PMcc @julian_knight and @keith_siilats
And this thread was pre Utility support on Hub: https://meltano.slack.com/archives/C013Z450LCD/p1644973255671469?thread_ts=1644973066.247049&cid=C013Z450LCD
j
julian_knight
06/09/2022, 5:08 PMWe’ve got it working, happy to help anyone who has questions
a
aaronsteers
06/09/2022, 5:16 PMThanks, @julian_knight! Do you have capacity to help us get it on the hub with a basic readme? Or if not, maybe we could block time for a brain dump session?
f
florian.hines
06/09/2022, 5:19 PMI haven't had a chance to look at how these integrations work, but just a heads up that in 2.0 the format of the
meltano schedule list --format=jsonj
julian_knight
06/09/2022, 5:41 PM@aaronsteers It’s wired up a little manually right now, for example we hard-code a
dbt snapshota
aaronsteers
06/09/2022, 6:57 PMYeah - that sounds great. And I know you get super busy. We can very likely assist with additional support in porting to 2.0 if needed. I'll open a 'Dagster smoke test runthrough' issue to assign to a Meltano team member whenever you're ready for us to kick the tires.
aaronsteers
06/09/2022, 6:59 PMAnd I keep saying this because I'm so excited about it: the new lock files feature means the spec we post to the hub does not need to be final. ❤️
Users who install it will get the best experience available at the time, and users who install later (or want to get the latest update later on) will get newer versions as/when they're ready for them.
j
julian_knight
06/13/2022, 1:51 PMI got a basic version using 2.0 and jobs working on a test project over the weekend. I'm sure when we run this in production there will be some issues to work out but it shouldn't be too hard to get this on the hub
https://github.com/rabidaudio/selfdata/blob/main/orchestrate/meltano.py
8 Views