Hi! I've been reading on Meltano and considering adopting it over Airbyte, but it would be necessary that I could configure extractors and loaders programatically from a Python application. This application would not be running in the same server as Meltano. The docs only mention the CLI, and there doesn't seem to exist an API. What would be the best way of controlling Meltano in this scenario?

Hi @Henrique Melo! The best option other than waiting for https://github.com/meltano/meltano/issues/2414, would be to call

meltano

in a subprocess, i.e. wrap Meltano commands in subprocess calls and expose those through your own API.

Thanks for the reply! It would be crucial to facilitate integrating Meltano with other tools that we could configure it through an API, and maybe that it kept its resources configurations in a database. yaml files could then be used as configuration declarations that could be `apply`ed to the server, so you'd still have your setup as code, if desired. I see that this is a different approach than what is currently being developed, but the way things work right now kind of blocks Meltano from being integrated with the rest of the data stack 😞

An idea - we have essentially exposed the meltano configuration via our UI and API. Presently we wrap all configuration an execute the meltano cli commands. However, directly writing the meltano.yml is my preferred future direction as we already automatically load / deploy meltano.yml changes (i.e. development the other direction where you change the meltano.yml and the UI is immediately updated) https://matatika.com/docs/getting-started/community-edition

Interesting! I'll take a look at your solution. Maybe it can fit right in with my application

Cool. DM me if you want to chat on a call.