hey there tappers, how do I do some.. safe logging while the tap is running? I realize my usual

print

method causes meltano or the singer spec to think I am outputting SQL data.. so I've erased everything.. but it would be nice to know for these long long jobs how far I am been looking over https://sdk.meltano.com/en/latest/implementation/README.html but not seeing anything about info or debug logging that can play nice with production I've got a tap running for 17 hours and it'd help to know if I can just.. print out where in the list of my queries am i

if you're using the sdk

<http://self.logger.info|self.logger.info>("log")

perfect, yes! Seems this needs to be documented so others don't start `print()`ing everything.. I'd wonder who I can maybe approach to start a how-to

Started https://gitlab.com/meltano/sdk/-/issues/170 a while back, I think it's similar!

interesting you've got PII data issues there as well

Mainly it's just about stderr shouldn't have PII, I think Immuta is a different thing, but I honestly don't know Immuta

we are using it for access .. and PII masking.. and I wonder outloud if / how that would work with the taps.. guessing it would modify the query the tap executes .. but is that capability out of the box

Access - You could use that catalog that's generated, but you'd need a layer above what already exists (and you'd need enforcement of that somehow), seems a lot better to put all the data into a warehouse, with the catalog, use that data to tell the warehouse who should have access to what with a tool like that. Putting it at the meltano level feels wrong but what do I know!