Great to see that we're getting Hub metrics regula...
# hub
t
Great to see that we're getting Hub metrics regularly now! https://github.com/meltano/hub/pull/632 @feather.rw do you know if we're rebuilding the Hub on a daily basis? @pat_nadolny has the Squared project pulling metrics daily so ideally we'd have it updated on the Hub once a day as well.
f
We aren't currently rebuilding the Hub site unless something changes in the source repo. There's no built-in way to trigger daily builds, but we might be able to do something ourselves using a github action on a schedule.
t
Good to know - I just made https://github.com/meltano/hub/issues/640 to track. It'd be worth getting something up and running for that I think.
f
I think there might be more of a conversation to be had around fetching the latest data dynamically rather than baking it into the static site. Depends on what our goals are for hub metrics going forward. Forcing this data into the static site paradigm will be a limiting factor if we want to expand on it or get handed tighter requirements for freshness. Some possible approaches with better long-term prospects: • The
metrics.yml
and
audit.yml
are in s3, and so it might be trivial to host those files publicly. Users visiting the site would be fetching the data files directly and they'd be getting fresh numbers as soon as the files are updated in s3 • AFAIK, these data files are being extracted from our data warehouse on an interval. It might make sense to rethink the extraction and find some way to expose the view/table as a small "hub data api". That way the metrics come directly from the source and will always reflect our current warehouse picture.
@taylor The issue you just created is a good starting point. But if we can initiate a med-term follow-up discussion, that'd help. I just don't want to lose the bigger picture (extracts and static site generation might be holding us back a little, while still taking effort to maintain)
t
Yeah, the way we're currently doing it could be a lot better for sure. Do you want to add some thoughts on https://github.com/meltano/hub/discussions/628 ? My goal with the daily re-build is to get us back to what we had right before 2.0. But in light of the Gridsome hub revamp and all that I'd love to get some more robust ways of doing this!