<#9506 feat: Parse structured logs coming from the...
# meltano-repo-activity
g
#9506 feat: Parse structured logs coming from the Singer SDK Pull request opened by edgarrmondragon ## Description ## Related Issues • meltano/hub#2114 • meltano/sdk#3279 • #9506 ## Summary by Sourcery Add structured logging support for Singer SDK in Meltano by introducing pluggable log parsers and integrating them into plugin output handling. New Features: • Introduce LogParser classes (SingerSDK and Passthrough) and a LogParserFactory for structured JSON log parsing. • Extend OutputLogger and Out to accept a log_parser parameter and attempt structured parsing before fallback logging. • Add get_log_parser method to ProjectPlugin and a _log_parser setting to enable per-plugin log parser configuration. Enhancements: • Refactor SingerPlugin logging setup to switch between basic and structured SDK logging configurations. • Propagate log_parser through plugin invocations and extract/load blocks for consistent handling. • Define a new constant LOG_PARSER_SINGER_SDK and use SettingDefinition for _log_parser in Singer plugins. Tests: • Add comprehensive integration and unit tests covering parser selection, parsing behavior, and fallback logic in OutputLogger and ProjectPlugin. ## Summary by Sourcery Add structured logging support for Singer SDK by introducing a pluggable log parser framework and integrating it into plugin output handling New Features: • Provide a log parsing framework with base LogParser, ParsedLogRecord dataclass, LogParserFactory, SingerSDK and Passthrough parsers • Allow OutputLogger/Out to accept a log_parser parameter and parse written lines before falling back to standard logging • Add a per-plugin _log_parser setting and PluginInvoker.get_log_parser method to configure structured parsing Enhancements: • Refactor SingerPlugin to emit structured or plain JSON logging configurations based on the selected log parser • Propagate log_parser through plugin_command_invoker, extract_load blocks, and I/O linking • Define LOG_PARSER_SINGER_SDK constant and register it in plugin settings definitions Tests: • Add unit tests for parser implementations, factory behavior, OutputLogger parsing and fallback logic, and plugin log parser configuration • Add integration tests validating end-to-end structured logging support in PluginInvoker and ProjectPlugin meltano/meltano