Does anybody have an experience of running meltano as python

subprocess

? I am trying to understand how it can be handled for long running jobs where log out put is high. Also there is no way to provide a config file for credentials so how can a

subprocess

pass in env variables for meltano cli to pick up

Crude example of setting the CLI log level by environment variable. Feels a bit wrong to be doing though...

Oh interesting, thank you Reuben

Nothing to do with passing environment variables to a subprocess, just that I find running and handling subprocesses to be a bit clunky - maybe you have more experience with that than I do though. 😅 I was going to say I'd rather use a library for this kind of thing, but obviously that isn't provided for Meltano currently as discussed in your thread from a couple a weeks ago, so I guess running a subprocess is your solution for the time being).

Yeah that sums it up 🙂

Cool, good luck then! 😁

and I am treading new waters running meltano sub process