Hi all, I have a question about `dbt-duckdb`: I have my meltano enviroment where I pull data from a tap and use the

target-parquet

to save it as parquet files, all well there. Then I want to bring dbt into the process, so I add it to the project (I'm new to dbt as well) and then I create my sources file:

sources:
  - name: awin-local
    meta:
      external_location: "{{ env_var('OUTPUT_PARQUET', 'data') }}/{name}-20241126_145148-0-0.gz.parquet"
    tables:
      - name: transactions

the OUTPUT_PARQUET is coming from the meltano.yml (I hope) so I expect this external source to point to my parquet file, I want to run queries against it using dbt

environments:
- name: dev
  env:
    OUTPUT_PARQUET: $MELTANO_PROJECT_ROOT/output/awin-parquet/transactions/
- name: staging
- name: prod

Now I try a simple model/all_data.sql

with trns as (
    select networkFee__amount from {{ source('awin-local', 'transactions') }}
)

select * from trns

and then I compile (

uvx meltano invoke dbt-duckdb compile

) to see the SQL and I get the following query

with trns as (
    select networkFee__amount from "main"."awin-local"."transactions"
)

select * from trns

and this fails to run (

invoke dbt-duckdb run

), what I'm expecting to see is something more like

with trns as (
    select networkFee__amount from "$MELTANO_PROJECT_ROOT/output/awin-parquet/transactions/{name}-20241126_145148-0-0.gz.parquet"
)

select * from trns

Based on my definition of the source and so that I can run queries against my parquet file. But I can't, and I've been unable to figure it out, any ideas? Thank you very much for your help 🙂

Linking here: https://github.com/duckdb/dbt-duckdb/issues/481. dbt doesn't seem to be picking up the source definition. Where is it located within the project?

Hi! Here is a screenshot of the file tree, so

sources.yml

is in the

models

folder I invoke the command from the same level as the

meltano.yml

I'll look into this tomorrow