Linen
Join Slack
*Adding `run_id`* to JSON logs When using a JSON formatter, the `run_id` isn’t specified in the res...
# troubleshooting
f
Adding 
run_id
 to JSON logs When using a JSON formatter, the 
run_id
isn’t specified in the resulting logs. Here’s an example line in 
meltano.log
⤵️
Copy code
{
  "consumer": true,
  "producer": false,
  "string_id": "target-snowflake",
  "cmd_type": "elb",
  "stdio": "stderr",
  "name": "target-snowflake",
  "event": "time=2024-03-14 16:41:30 name=target_snowflake level=INFO message=Loading into REDACTED.\"REDACTED\": {\"inserts\": 315600, \"updates\": 0, \"size_bytes\": 13682350}",
  "level": "info",
  "timestamp": "2024-03-14T16:41:30.459033Z"
}
Therefore, when exporting to Datadog, I can’t distinguish my different jobs. Is there a way to fix that? I put my 
logging.yaml
file in the thread 🧵
Copy code
# logging.yaml

version: 1
disable_existing_loggers: false

formatters:
  structured_plain:
    (): meltano.core.logging.console_log_formatter
    colors: True
  json:
    (): meltano.core.logging.json_formatter

handlers:
  console:
    class: logging.StreamHandler
    level: WARNING
    formatter: structured_plain
    stream: "<ext://sys.stderr>"
  file:
    class: logging.FileHandler
    level: INFO
    filename: meltano.log
    formatter: json

root:
  level: INFO
  propagate: yes
  handlers: [console, file]
e
Hey @florian_ernst! I think we'd just need to bind a 
run_id
property to the logger in https://github.com/meltano/meltano/blob/74d826f3ccbe03ccc986fafa229dec7cdcaab782/src/meltano/core/block/extract_load.py#L607-L612. PRs welcome!
11 Views
Open in Slack
PreviousNext
Powered by

Meltano

Meltano community

Powered by