Hi! I am looking at scheduling/orchestrating options, and it would be nice to go with simple cron jobs and avoid the airflow installation (with the db, users, ui, etc). At the same time, having a UI to run jobs manually when necessary is useful (not everyone has access to the CLI 😅) This is currently possible with the meltano-ui (which seems simpler to set-up and to use), but as I understand it, there is no “cron orchestrator”, only the airflow orchestrator? So schedules defined in meltano.yml would be visible and runnable via the UI, but without Airflow they would not be run periodically (i.e. the

interval

config param would be useless)? Would this be a good idea, or do you have big changes planned ahead regarding the “Schedules” meltano ui?

Good question @alex! There’s a couple of things tied together in this question. I think the first one is that we could definitely have a simple “cron orchestrator” in lieu of Airflow - some of the challenges with this though come in deployment - do jobs get run in the same container / pod that Meltano is deployed in? For a lot of things that could be ok, but other it might not be. As for the UI - we’re working to hire an additional engineer with frontend experience so we can continue to add more to the UI to bring more features from the CLI into the UI. So yes, I think it’s a good idea. Would you mind making an issue around this?

Thanks! Indeed, for the “cron orchestrator” the use case would be a simple VM running meltano (no K8s, not even docker in my case) I just created the issue: https://gitlab.com/meltano/meltano/-/issues/3522 🙂