I just did a "Meltano in Action" demo on the evolution of my hack day project
https://www.youtube.com/watch?v=pNGJ96HOioM▾
. TLDR; its a new iteration on our Airflow dag generator that tightly integrates with dbt and translates dbt dags to airflow dags, it will try to run an elt sync for each source in your dbt project. I have this MR https://gitlab.com/meltano/files-airflow/-/merge_requests/8 up for review and feedback which contains all the logic for the new functionality, check it out. I'd still consider it beta (so use with caution) but I've worked out a bunch of bugs already.
a
aaronsteers
03/24/2022, 8:42 PM
This is exciting! 🎉 Thanks, @pat_nadolny for creating and sharing.
aaronsteers
03/24/2022, 8:44 PM
If anyone else is viewing immediately, note that YouTube lazily adds HD resolutions. Right now only
360p
is available, but higher definition should be available within the next few hours. 🙂
p
pat_nadolny
03/24/2022, 8:59 PM
Thanks yeah I've been cleaning up my hack day project slowly and I think its finally in a shareable state, hoping some people get excited about it and give me more ideas of how to make it better 😁. I'll probably try to get it configured in the Squared project to see what challenges come out of that