joshua_janicas
07/24/2023, 1:36 PMtarget-snowflake (Meltano Variant). Snowflake would have one distinct warehouse (and set of DBs) per client to ensure separation of information. I am hoping to have just one instance of Meltano/DBT running instead of needing to configure one per client as well, with likely some kind of orchestrator in the background (Airflow? Dagster?).
While each client is going to have the same general loader/extractor settings, the rub is that each tenant is not identical in their schema. Example, client 1 may have a more up to date version of our database software than client 2 (who is scheduled for deployment in the future), and has some different schema due to said software updates.
Would having a different yml per client for meltano to reference work? Feeding different YML files to meltano? Is this setup only possible with one meltano/dbt/orchestrator per client to ensure distinctness?
Any other thoughts or examples of this attempted before?
Attached is a professional ( 😂 ) diagram to visualize what I am trying to do here.joshua_janicas
07/24/2023, 1:38 PM