Anyone dabble with using meltano in an Athena cont...
# plugins-generala
andrew_stewart
03/28/2021, 12:25 AMAnyone dabble with using meltano in an Athena context? (I imagine that mostly taking the form of s3 targets)
a
aaronsteers
03/28/2021, 12:28 AMNot personally but I'd love to hear from others if they have tried it! @andrew_stewart, are you also in dbt slack? There's a channel for Athena as a dbt backend so this certainly should be doable! https://getdbt.slack.com/archives/C013MLFR7BQ/p1616772168010200?thread_ts=1616772168.010200&cid=C013MLFR7BQ
a
andrew_stewart
03/28/2021, 12:34 AMYup, that was me, asking on the other side of the issue. I feel like it’s a pretty intriguing idea, esp if one realizes the constraints of athena as a dw engine.
a
aaronsteers
03/28/2021, 12:53 AMOh great! And yes, I love this idea! It also fits well with a new initiative we are discussing for Meltano, which is to ship with one or more “bundled” data engine backend options. @andrew_stewart - I’m curious to hear your thoughts on this thread, if you are thinking in a similar direction: Explore potential for out-of-box data repository, data lake, and/or SQL interface layer (#2634) · Issues · meltano / Meltano · GitLab
a
andrew_stewart
03/28/2021, 1:00 AMOh, fantstic idea.
andrew_stewart
03/28/2021, 1:01 AMI’ve been writing a series of articles and tutorials on learning analytics stacks + data engineering basis using open source components (https://github.com/andrewcstewart/ds4fnp) - so far most of it just using local postgres or sqlite, but some columnar store out of the box would be interesting.
andrew_stewart
03/28/2021, 1:02 AMI’ll also be interested to see which options have been suggested here. It’s hard to keep up with that part of the stack.