Hey everyone 👋 Question for yall -> Has anyone ever succesfully removed the prefix "METRIC" from a singer SDK metric? I'm trying to implement full json logging and it's a bit difficult 😆 https://sdk.meltano.com/en/v0.43.1/implementation/logging.html#custom-logging-configuration I'm looking into

custom_logging_configurations

, but the

METRIC:

seems to be part of the log -> I can't remove it without creating a filter. Any ideas?

Hey @Stéphane Burwash, glad to see you here! Got an old PR to help with that: https://github.com/meltano/sdk/pull/2162. I'm curious, can you say more about your use case? Like what sort of pipeline you have in place for logs, whether you're using a platform or framework (e.g. Datadog, OTel, etc.). I'm trying to think how to make metrics easy to ingest for a majority of folks, and this would help inform the design. Thanks!

Ouh goody! Do I need to wait for you to merge that, or can I apply it myself?

> Do I need to wait for you to merge that, or can I apply it myself? Unfortunately, you'd need to wait for the metric json to be accessible with the need to parse the

METRIC ...

string. > Currently we're using Elastic -> I could also parse the logs (currently what I'm looking into), but would love to go to the source if I can > I would like better access to the metrics so that I can investigate anormalities & bugs, implement alerting, etc. etc. Ok, that makes sense. It's in line with the approach I'm taking. > I'm trying to implement full json logging How are you dealing with other Singer tap logs? i.e. non-metric messages

Using the

log-format=json

feature, but beyond that not much -> We're just consuming them as-is.

Gotcha

So until you create the

metric_json

, easiest would be to parse the logs before ingestion in elastic (using someting like vector)?

Correct. Related to this, I have a project that I've never really had time to move forward 😅 but it's a Singer metrics parser for the Influxdb line protocol: https://github.com/edgarrmondragon/singer-metrics/. Maybe it's a helpful reference.

I'll look into that tomorrow, thanks!