What’s the best way to have the Airflow scheduler ...
# troubleshootings
saadat_qadri
05/25/2021, 6:12 PMWhat’s the best way to have the Airflow scheduler running in a containerized Meltano project? My current setup doesn’t use docker-compose - this is what I’m trying in the Dockerfile but it’s not working:
Copy code
# Start the aiflow scheduler in the background.
RUN meltano invoke airflow scheduler -D
RUN meltano invoke airflow webserver -p 8080 -Dsaadat_qadri
05/25/2021, 7:10 PMsorry if this doesn’t make the most sense. I can’t seem to have the scheduler running in prod. I can run the scheduler in the foreground, and that picks up the etl jobs no problem.
d
douwe_maan
05/25/2021, 7:11 PMCommands in the dockerfile will only run as part of the build process, they won’t be running automatically once you deploy the container. What are you using to orchestrate your Docker containers if not Docker-compose? That’s where’d you want to say “use this container and use this as the initial command”
s
saadat_qadri
05/25/2021, 7:17 PMah that totally helps @douwe_maan. I have a simple shell script that spinning up the container on AWS Fargate, but nothing beyond that to specify what commands to run
saadat_qadri
05/25/2021, 7:17 PMseems like docker-compose is the way to go
d
douwe_maan
05/25/2021, 7:24 PM@saadat_qadri Fargate should have a way to specify an entrypoint comment
douwe_maan
05/25/2021, 7:24 PMOr your script that runs
docker run