Is there a way to prehook the meltano run command?...
# plugins-general
j
Is there a way to prehook the meltano run command? I want to initialize google cloud logging before pipeline runs. Right now the output is all labelled as errors. I am looking into the Meltano EDK right now
Copy code
# Imports the Cloud Logging client library
import google.cloud.logging

# Instantiates a client
client = google.cloud.logging.Client()

# Retrieves a Cloud Logging handler based on the environment
# you're running in and integrates the handler with the
# Python logging module. By default this captures all logs
# at INFO level and higher
client.setup_logging()
e
Hi James 👋 It's not something supported at the moment, but do log an issue if this is something crucial to your use case! That said, you might be interested in this gist, which configures the ops agent to gather logs coming from Meltano. In a previous revision of the same gist, I was using a custom handler function and its directory to
PYTHONPATH
so Meltano could import the module it at runtime: https://gist.github.com/edgarrmondragon/feb148d50339dbbc2a6251707e13882f/revisions.