Has there been any community attempt to add Open T...
# random
b
Has there been any community attempt to add Open Telemetry support in some fashion into Meltano to provide a way for us to scrape out the metrics that are produced on an
elt
run? I see helpful metrics such as
record_count
and
sync_duration
in my task logs for the run, but apart from scraping the metrics out of the logs there doesn't seem an intuitive way to pull out metrics into a collector at the moment.
I am surprised there isn't a table in the state DB for this information either, seems like it would be a nice-to-have as then I could just poll these metrics out of my Postgres Meltano state DB
I got it working without too much difficulty - there’s no instrumentation for the
meltano
innards but you can use the autoinstrumentation libraries for things like
requests
and
sqlalchemy
I agree with you that this would be super useful to have in meltano proper
b
Oh nice! Thanks, I can definitely bootstrap this, same pattern we are using for some other apps that don't directly expose a hook for opentelemetry
m