Hello All I have deployed my Docker Image of Meltano to AWS Meltano #troubleshooting

Hello All, I have deployed my Docker Image of Mel...

steve_clarke

12/06/2021, 3:55 AM

Hello All, I have deployed my Docker Image of Meltano to AWS Container Registry (ECR). I have also enabled scanning of the image for vulnerabilities. I note there are many security vulnerabilities listed, and they appear to be mainly attributed to the docker python base image. Having come across this article, Snyk Python Docker Best Practices, they appear to recommend alternative base images for Python. For example I am building a Meltano Python 3.8 Image, the recommendation is

Copy code

python:3.8.12-slim-bullseye

I'm wonder if it is possible to have alternative Meltano Docker builds which I could pull from with a more hardened base image? I can't comment however whether this image will run Meltano correctly or not. I do know however that my security team would be a lot happier 😀. I also have another related security question regarding best practices for environment variables. Really keen to get a steer on that too. https://meltano.slack.com/archives/C01UTUSP34M/p1638488867102100 Thanks Steve

steve_clarke

12/07/2021, 7:24 AM

A small update, I have had a go at preparing my own custom Docker Image based on a custom python base image. I'm not sure if I have gone overboard or not. It would be good to have some comment on this. I have followed the Meltano approach of two Docker builds one for the base and one for the particular container build. Thanks Steve

Dockerfile - Prod Dockerfile - Base

edgar_ramirez_mondragon

12/07/2021, 6:35 PM

Hi @steve_clarke. Thanks for sharing! Those dockerfiles look good. Have you had a chance to scan the Meltano image based on

python:3.8.12-slim-bullseye

? If the slim images work this may be something we want to implement on the Meltano images in dockerhub

steve_clarke

12/08/2021, 5:03 AM

Hi @edgar_ramirez_mondragon, Yes the recommended Snyk images improve things a lot. My docker image based on the

python:3.8.12-slim-bullseye

image has reduced the number of vulnerabilities by quite a bit. I do think these slim images might be good, they also make smaller images which is good. I shaved off 200MB with the slim image. I probably could improved this with refactoring my docker image a bit. See the attached table for the scan difference. If you were able to use the slim python images for the base that would be much appreciated. Thanks Steve

steve_clarke

02/22/2022, 10:29 PM

Hi @edgar_ramirez_mondragon, @aaronsteers, Just wanting refresh the conversation regarding possible variant docker image builds of Meltano using a different Python Base. My security team wanted us to reduce the number of vulnerabilities that are picked up by container scanning. Swapping out the standard python base to the slim versions reduces both the size of the docker image but also the number of vulnerabilities. In my mind, it would seem quite feasible create an additional meltano image build using slim versions. Thoughts? Thanks Steve

aaronsteers

02/22/2022, 11:12 PM

@steve_clarke - Do you know if this has been logged yet as an issue? And if not, do you mind logging this in our issue tracker? I think we could approach this as (1) replacing the existing builds or (2) adding on new image versions.

steve_clarke

02/23/2022, 7:58 AM

Thanks @aaronsteers, I have raised an issue now. https://gitlab.com/meltano/meltano/-/issues/3278 . Please let me know if you need any further details.

aaronsteers

02/23/2022, 4:44 PM

Perfect, thanks! I've actually added this into our #C01QS0RV78D board so we'll get a quick chance for team feedback in about 15 minutes. You are welcome to join, but not required. :)

aaronsteers

02/23/2022, 6:02 PM

We ran out of time, sadly, before we could really dive in. (Lot's of other great topics though.) We'll continue discussion in ascync there on the issue.

Open in Slack

Previous Next