Hi, Is there any way to decrease docker image size when there are too many extractors in Meltano project ?

A few options I can think of are: • if youre installing the plugins at build time then consider if youre installing anything thats not needed i.e. dev utilities, etc. and exclude them • consider installing the dependencies at runtime using something like

meltano install --schedule=<schedule_name>

https://docs.meltano.com/reference/command-line-interface#install. Its slower to startup and is more susceptible to startup failures because it has to run a pip install

I was thinking about this. Btw. some extractors/loaders are much bigger than usual, AVG size is circa 20M, but size of target-snowflake is 400M! How many combinations of extractors/loaders do you need to support?

Currently I have 10-12 extractors/loaders and docker image size is around 600MB. Half of them are based on RestAPI and have same dependencies.

Just so I understand, in my dockerfile I currently have

meltano install

as per the std file set. But I deploy my image to Azure Container Apps, which means I start the image with CMD

meltano invoke dagster

Could I use

meltano install --clean && meltano invoke dagster

as the CMD and reduce the dockerfile size considerably? The loop from one line change to pushing to my container registry is very slow at the moment, that would be great. Its the

meltano install

command that populates the venvs in

.meltano

dir correct? That folder is normally listed in

.dockerignore

which would make sense