Hello all, I want to use dbt-snowflake to create s...
# troubleshootinga
adam_mustapha
12/07/2022, 1:49 PMHello all, I want to use dbt-snowflake to create schemas on my local computer and test functionalities in isolation, I have watched so many videos but haven't figured out a way. I'll appreciate any help. Thank You
p
pat_nadolny
12/08/2022, 2:59 PMHey @adam_mustapha! We do this in the squared project and I created an overview in this . But to answer your question, theres no way to run a local instance of Snowflake on your laptop (as of today) so we use separate databases/schemas with distinct users/roles in our snowflake account to allow for isolation. My usual workflow is to test my changes against my personal dev environment (still connecting to Snowflake but using my personal user and role with limited access), then create a PR that is tested against an isolated CI environment in snowflake (connects to Snowflake using a CI user/role), then its ultimately deployed to my live production databases/schemas.
pat_nadolny
12/08/2022, 3:02 PMAll of our non-production environments have their own set of snowflake databases and schemas. For example in production we have RAW/PREP/PROD databases then in our user isolated environment we have USERDEV_RAW/USERDEV_PREP/USERDEV_PROD and similar with CICD_RAW/CICD_PREP/CICD_PROD. Meaning 9 databases in our snowflake instance in order to support this workflow.
pat_nadolny
12/08/2022, 3:39 PMTheres a lot of different ways to do this so you can chose to customize how you want. For example another approach is using 1 database per environment and using schema isolation using naming prefixes. A way to do this is using a generate_schema_name macro in dbt (I borrowed this from GitLabs data team's implementation)