A couple things I have learnt from trying to deploy Meltano on Azure Container Instances (ACI) via bicep, hopefully they can be of use to someone else. • When creating the ACI, you need to define your

meltano

command, and it cannot be altered. There is no way (I've found) to run an image with a custom command, it has to be fixed along with the container at creation time. So you end up having one container for each tap & target combo, which leads to... • Very quickly hitting the default Azure quota for assigned Container vCPU, which is 10 for your whole region. So if you have 10 tap-target combo ACIs created and you assign them 1 vCPU each, you can't create any more ACIs. Even if those images are not running, they count as 'assigned' for vCPU quota purposes. So I have scaled down to fractional vCPU for many of them to allow me to create some more. I would make a request to increase the vCPU limit immediately as requires Azure support to do that I have kind of got to a reasonable working solution, but feel like this would be much easier on AWS 😞

For point 1: Or you can have a simple flask app that accepts commands ans just triggers the command

if you use azure container apps would it help the situation?

Yes I use ACA now 🙂

Can you try azure container instances?

I did use ACI initially (hence my first message), but quickly moved to ACA.