Seeing some weird log formats: From DBT:

[32mCompleted successfully[0m

From tap-mysql:

time=2022-02-25 00:15:46 name=tap_mysql level=INFO message=Stream ... is using incremental replication

Is there a way to have Meltano inject its log configuration into the taps/targets/transforms? It looks like

singer.get_logger

lets us override the tap/target log config, is that wired up in Meltano? I don’t see it with a cursory search. Seems it defaults to

<tap_dir>/logging.conf

by default. Should Meltano copy its own logging.conf in to taps/targets when installing them?

Hi @paul_tiplady! I'll try to answer your questions:

It looks like 

singer.get_logger

 lets us override the tap/target log config

That is only true for taps/targets using

transferwise/pipelinewise-singer-python

, not

singer-io/singer-python

. The SDK does not (yet) use

singer.get_logger

, so SDK-based packages do not support it. This could make a good issue if you'd like to file it 🙂

is that wired up in Meltano?

Not at the moment. But, if the tap uses pipelinewise's library logger, you should be able to configure the logs by exporting a

LOGGING_CONF_FILE

env var pointing to a custom file similar to https://github.com/transferwise/pipelinewise-singer-python/blob/master/singer/logging.conf

Should Meltano copy its own logging.conf in to taps/targets when installing them?

You mean a custom one, or reuse

logging.yaml

somehow?

I created this one in Meltano, since I think the meltano side is a superset of concerns — one possible solution would be an SDK fix, but unless the plan is to make Meltano SDK verisons of all taps, then that wouldn’t fully solve the problem.