Hello Channel! I am getting an error while running the meltano airflow webserver and scheduler on AWS ECS (Fargate). I get this error:

Error: No such command 'invoke airflow scheduler'
Try 'meltano --help' for help.
Usage: meltano [OPTIONS] COMMAND [ARGS]...

meltano UI works okay. The problem is only with airflow scheduler and webserver. I am hosting my built image in ECR registry. I looked at the built logs and I can see that airflow plugin was installed correctly. Here is the log of plugin installation during the build before the docker image was pushed to ECR.

Installing 4 plugins...
Installing file bundle 'airflow'...
Run `meltano upgrade files` to update your project's 'airflow' files.
Installed file bundle 'airflow'

Installing orchestrator 'airflow'...
Installed orchestrator 'airflow'

Installing loader 'target-redshift'...
Installed loader 'target-redshift'

Installing extractor 'tap-hubspot'...
Installed extractor 'tap-hubspot'

Installed 4/4 plugins

on the meltano UI on ECS, I can see that the plugins tap-hubspot and target-redshift are listed correctly, so I guess airflow plugin would have been installed as well (because the Dockerfile has a single command

RUN meltano install

to install every plugin listed in my meltano.yml file) I am still trying to debug what has gone wrong with airflow scheduler and webserver. And I tried replacing the command

invoke airflow scheduler

with the full command

meltano invoke airflow scheduler

but I get this error:

Error: No such command 'meltano invoke airflow scheduler'.
Try 'meltano --help' for help.
Usage: meltano [OPTIONS] COMMAND [ARGS]...

Looking at the error, I guess the meltano bin file is correctly in the system PATH, but it is not taking the args correctly. Did anyone else saw this error? (not necessarily on ECS, even if you saw this error in your local machine, it is worth identifying the root cause and fixing it for ECS).

@pankaj_saini It looks like

invoke airflow scheduler

is being passed to

meltano

as a single argument, rather than separate

invoke

airflow

scheduler

arguments that are then interpreted as command

invoke

plugin

airflow

args

scheduler

🤦‍♂️ How did I miss that? You beauty! I am trying a list of commands now instead of a single command.

😊

Worked like a charm!!! ❤️

Woo!