Hello everyone, I have a question - Using meltano, I am able to configure requirements of a single user and fetch relevant data for that user via an extractor ( be it tap-gong, tap-salesforce etc). But suppose we want to create a platform, where multiple users can come and select multiple extractors as source for their data and we will load the data and show in the UI. I guess my question is, can I configure the meltano pipeline for gong/salesforce extractors to fetch data of multiple users wherein each user can have different configurations. I will be storing configuration for each user from into a database and meltano pipeline will pick the details for each user one-by-one or in parallel. What would be the best way to achieve this?

Your best option at the moment is to make this happen at the orchestrator level with Airflow, k8s, ECS or whatever orchestration tool best fits your current stack. You could have a script retrieve each user's settings from the db and spin up a Meltano container or CLI task for each user, passing each setting as the corresponding environment variable Meltano expects.

Hi @Sakshi Dangi - agree with Edgar. This is pretty much exactly what Matatika does for each workspace - a group of users rather than user. Same idea though.