Can you configure the dbt transformer somehow that it doesn’t prepend the schema set in meltano.yml to every schema?

Hi @Johan Vrolix! What have you tried so far?

Apart from omitting the schema in the meltano configuration and setting schemas in the dbt project yaml, nothing really comes to mind

Ok, so how do your

meltano.yml

and

dbt_project.yml

look like?

meltano.yml

version: 1
env:
  DBT_CLEAN_PROJECT_FILES_ONLY: 'false'
default_environment: dev
project_id: e7660772-c6df-4e04-9a94-21281710aba6
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-postgres
    variant: meltanolabs
    pip_url: git+<https://github.com/MeltanoLabs/tap-postgres.git>
  - name: source-northwind
    inherit_from: tap-postgres
    config:
      host: localhost
      user: northwind
      database: northwind
      port: 5435
    select:
    - public-shippers.*
    - public-suppliers.*
    - public-customers.*
    - public-employees.*
    - public-orders.*
    - public-order_details.*
    - public-products.*
    - public-categories.*
    - public-territories.*
    - public-employee_territories.*
    - public-us_states.*
    - public-customer_demographics.*
    - public-customer_customer_demo.*
    - public-region.*

  - name: tap-rest-api-msdk
    variant: widen
    pip_url: tap-rest-api-msdk
  - name: ellie-metadata
    inherit_from: tap-rest-api-msdk
    config:
      api_url: <https://optimalist.ellie.ai>
      streams:
        - name: metadata
          path: /api/v1/models/$ELLIE_ID
          primary_keys: ["model_modelId"]
  loaders:
  - name: target-postgres
    variant: meltanolabs
    pip_url: meltanolabs-target-postgres
  - name: target-dwh
    inherit_from: target-postgres
    config:
      host: localhost
      user: dwh
      database: dwh
      port: 5433
      threads: 4
  utilities:
  - name: dbt-postgres
    variant: dbt-labs
    pip_url: dbt-core dbt-postgres meltano-dbt-ext~=0.3.0
    config:
      host: localhost
      port: 5433
      user: dwh
      schema: dwh
      database: dwh

So what's the problematic prefix?

The ‘dwh’ schema configured in meltano.yml gets prefixed when building the dbt models. Eg. For the raw_datavault objects the schema becomes dwh_rdv

We just overrode dbt’s builtin

generate_schema_name

macro:

{% macro generate_schema_name(custom_schema_name, node) -%}
    {%- set default_schema = target.schema -%}
    {%- if custom_schema_name is none -%}
        {{ default_schema }}
    {%- else -%}
        {{ custom_schema_name | trim }}
    {%- endif -%}
{%- endmacro %}

So next release?

Yes, if you define your own macro with the same name as a builtin macro, your locally defined one will take precedence.

Cool, thanks