https://linen.dev logo
#singer-tap-development
Title
# singer-tap-development
t

tonci_kokan

02/01/2022, 9:31 PM
Hi all, We’ve built several custom taps using Singer SDK and we’ve got an issue we’re unable to solve. We’re emitting logs from our taps and they are being captured by Meltano like this:
Copy code
[2022-02-01 20:47:38,510] [79340|MainThread|meltano.cli.elt] [INFO] {'run_id': 'e47bc532-998c-4f85-8d3a-273a0ce33bbd', 'job_id': 'tk-test', 'stdio': 'stderr', 'cmd_type': 'extractor', 'name': 'tap-bw-ga-experiment-data-fr', 'event': 'time=2022-02-01 20:47:38 name=tap-google-analytics level=WARNING message=some warning', 'level': 'info', 'timestamp': '2022-02-01T20:47:38.510142Z'}
[2022-02-01 20:47:38,510] [79340|MainThread|meltano.cli.elt] [INFO] {'run_id': 'e47bc532-998c-4f85-8d3a-273a0ce33bbd', 'job_id': 'tk-test', 'stdio': 'stderr', 'cmd_type': 'extractor', 'name': 'tap-bw-ga-experiment-data-fr', 'event': 'time=2022-02-01 20:47:38 name=tap-google-analytics level=ERROR message=some error', 'level': 'info', 'timestamp': '2022-02-01T20:47:38.510404Z'}
The problem is that Meltano emits all logs with
INFO
level, instead of mapping log levels from taps (
level=WARNING
and
level=ERROR
in the example above). Is it possible to map log levels or otherwise prevent Meltano from wrapping all stderr output in info logs?