Happy Friday everyone, I'm trying to monitor Meltano runs and have come across the tap-meltano plugin - sounds like it should do exactly what I need, but I'm having some trouble getting it to work (my config is minimal, just specifying the uri for the sqlite file, but getting the error: ValueError: invalid literal for int() with base 10: '') Are there any best practice approaches for reporting on job statuses? My approach is to try to get the contents of the .meltano database into Snowflake so I can use Tableau to help the team see what's happening. Is using the tap-meltano plugin a "good" approach? Cheers, Andy

@Matt Menzenski or @visch may have some good ideas here 😄

We’re on our second Meltano implementation. In our first we used tap-meltano (I remember having to fork it to support Python 3.11) but we never really did much with that data. Today we just use tap-postgres to load the Meltano system database data (runs and state) into Snowflake. We run Meltano in Argo Workflows and also load the Argo workflow archive data using tap-postgres. Between the two data sets we have a pretty full accounting of execution history. I would like to install wal2json in our Postgres instance and start using the log-based replication of tap-postgres (to be able to capture state over time) but haven’t set that up yet.

Thanks for your view @Matt Menzenski - moving to a Postgres backend for the Meltano database sounds like a sensible approach, so I'll do that!

Hi @Andy Crellin, sorry you ran into issues with the tap - it's been gathering dust since breaking with Meltano v2 a while back. I spent the weekend getting it working again (including adding support for current Python versions), so if you want to give it another go:

meltano install --clean extractor tap-meltano

Couple of extra things: • Config for the database URI is no longer required, as it will default to the working directory Meltano project SQLite system db • Can't see where

ValueError: invalid literal for int() with base 10: ''

would be coming from so it could be your URI config, so try omitting it in favour of the default, or otherwise feel free to open an issue with the full error trace (or post here)

Hi @Reuben (Matatika) - absolutely amazing! Thanks for coming back to this and working on it (what a community!). I will give this a go today and let you know how it goes. Many thanks (and think the error I was getting may just be a red herring) Andy.

Great! 😄