Hey folks, We're building a data connector flow that will allow user to authenticate our app for Sharepoint integration, in which we will ingest & regularly sync the files from their sharepoint into our own datasource. The thing is each user will have different Sharepoint (maybe Sharepoint online or on-prems), they will do oauth flow on their own. -> My question is: how can we dynamically add new extractors based on user request, since i see that Meltano is supporting add extractor via CLI only, not API. Do we have to build some backend API to receive new credential & executing some shell script underneath to add extractors into pipeline? And if that's true, will that requires us to restarting the docker that we're running Meltano on. Thanks in advance! Really appreciate your product!

If you mount the

plugins

directory on a volume, and run

meltano add

outside of the container, you can probably get away without restarting the Meltano container. That said, it probably doesn't too much make sense to have Meltano run in a long-lived docker container since there's no service to be run there.

Thank you for explaining me that. For the first question, that means we do have to implement this mechanism in our usecase, right?

Do we have to build some backend API to receive new credential & executing some shell script underneath to add extractors into pipeline?

Will you be not using a admin account to sync all files from share point to your data source . Are this like individual accounts which will be managed at user level . I will recommend using admin access and try if possible than going through each user and adding use case user by user will be maintenance overhead . If you have no option my recommendation is to do this • store all you users and related creds in a table • before you start Meltano el . Write a program to loop through this creds and pass them to Meltano

Got your point. Thank you guys!