Hey everyone - we (Brooklyn Data Co) have just ope...
# infra-deployment
n
Hey everyone - we (Brooklyn Data Co) have just open-sourced our Meltano on GitHub Actions template. Here are some reasons we love this approach: • Requires only a GitHub account to deploy a fully automated Meltano project with logging, alerting, and incremental state • Configuration is version controlled and maintained as code • Low barrier to entry, a working project can be deployed on a schedule in a couple of hours • Easy management of secrets • Low cost. See the free minutes for your plan here, and the incremental cost outside of your free minutes here. There are some limitations which may mean it's not suitable for your use-case: • There is a 6 hour maximum run time for any isolated individual job in a workflow. This often means that backfills need to be performed outside of GitHub actions. • No way of exposing the Meltano UI The template has built-in config for Snowflake and BigQuery destinations currently, but contributions are very welcome! Take a look here: https://github.com/brooklyn-data/meltano-on-github-actions
a
So cool, @niall_woodward! Thanks for creating and thanks also for sharing. I'm adding a TODO issue for us to review and see how Meltano can support/contribute here.
n
Thank you @aaronsteers!
c
Hi @niall_woodward - This looks quite interesting, but I noticed there is no activity in the GH repo. Are you still using this (perhaps in a private repo), and if so, how is it working for you?
n
We do still use it with lots of clients, it works great @chrish
The Meltano version is <v2 so might be worth updating the docker file and any CLI args in the workflows though.
c
Thanks! I've been debugging an issue with building my meltano Docker image for a couple of days, but now that I've got it figured out I should be ready to take a look at this in the next week or so.