Heya! I'd like to put my containerised meltano elt pipeline on AWS lambda, but our ops department insists on using assume role in order to fetch the access key, secret and the security token. Which is reasonable.. however it seems most (if not all) taps / targets do not support such a thing. Is there some mechanism in Meltano that does the assume role => exports credentials at runtime? Or do I need to add the boiler plate for it in the

lambda_handler

function? If so, are there some singer spec guidelines around this or some best practices? The way I see it, I'd need to fork every tap/target and modify it to take advantage of the IAM roles 😕 Anyone got any experience or tips arounds this?

@janis_puris - So, first, just full disclosure that running on Lambda will probably be an uphill battle. Most taps and targets are not optimized for short runtimes, and (because of the runtime limitation) you won't find a lot of implementation examples out there in the wild. That said, if the concern is just about having access to AWS resources when running with 'assume role', many/most components that connect to AWS resources like S3 will use the standard

boto3

library (or similar) internally, and they should pickup your default/activated AWS profile from context as long as you don't also provide a new set of credentials via config.

We went with containers and boto3's session() Worked like a charm! Thank you!!

Awesome! So glad to hear it!