Is anyone orchestrating meltano Meltano with P Prefect? 🙏 Would absolutely love to hear how this is achieved i.e. containerise the Meltano project and run that as a task in Prefect or run some sort of python wrapper for the CLI.. or other way.

From memory, yes. I think there's at least 1 active community member who does. I can't remember who ... I'm sure they will chime in. Or if somebody else remembers who it is, they'll cc them.

Any of @Henning Holgersen @david_wallace @lidong_zhao @boggdan_barrientos I think are running on some other orchestrator. You can find @Henning Holgersen prefect workflow here: https://github.com/radbrt/prefect_flows/tree/main/projects/jottings/meltano_flow.

I can explain this more later today, but some old code that illustrates my approach: https://github.com/radbrt/prefect_flows/tree/main/projects/jottings/meltano_flow Sadly it doesn’t run anymore, some repos have moved.

Awesome! Thank you so much! 💚

We are testing prefect, we run Meltano inside a container executed with fargate and ECS Task in prefect.