has anyone used `sqlfluff` with dbt models managed by meltan Meltano #best-practices

has anyone used `sqlfluff` with dbt models managed...

juan_luis_cano_rodriguez

06/28/2022, 7:23 AM

has anyone used

sqlfluff

with dbt models managed by meltano? I tried but I got a bunch of errors, so I'm asking before I get too deep into the rabbit hole 🐇

alexander_butler

06/28/2022, 12:50 PM

Yeah works great.

edgar_ramirez_mondragon

06/28/2022, 1:58 PM

Here's an example: https://github.com/edgarrmondragon/meltano-dataops/blob/main/meltano.yml#L130

pat_nadolny

06/28/2022, 3:29 PM

Here another example https://github.com/meltano/squared/blob/cdbfc6a15ae6dd9e6c3f3b4baca94b0159ddcd1b/data/utilities/utilities.meltano.yml#L3. @alexander_butler recently had the idea to alter the namespace to

dbt_<adapter_name>

so sqlfluff can override dbt env vars. Not sure if the same issue youre having but my meltano project was complaining that dbt env vars werent set when I ran sqlfluff, this namespace change fixed those.

juan_luis_cano_rodriguez

06/29/2022, 3:52 PM

oh, I didn't know about utilities! they're not mentioned in the "getting started" guide 😅 this is exactly what I needed, thanks folks! will give this a try soon

vadiem_janssens

08/15/2022, 2:09 PM

Hi all, hopefully this is still being red by someone. I've added the following code to my

meltano.yml

file, howerver I can't get sqlfluff to work.

Copy code

utilities:
    - name: sqlfluff
      variant: sqlfluff
      pip_url: sqlfluff[dbt]==1.2.1 sqlfluff-templater-dbt==1.2.1
      executable: sqlfluff

vadiem_janssens

08/15/2022, 2:10 PM

After that, I run

meltano install

and

meltano invoke sqlfluff lint -v

, but I keep getting the following error

Copy code

Starting in sqlfluff version 0.7.0 the dbt templater is distributed as a separate python package. Please pip install sqlfluff-templater-dbt to use it.
Error loading config: Requested templater 'dbt' which is not currently available. Try one of raw, jinja, python, placeholder

juan_luis_cano_rodriguez

08/15/2022, 2:13 PM

@vadiem_janssens you might have more luck opening a new thread, otherwise only people that participated here will read it

pat_nadolny

08/15/2022, 2:29 PM

@vadiem_janssens I think you need to add

dbt-core

to your pip url also like https://github.com/meltano/squared/blob/6271f7d9b987c128114dc376618b434e1929b8e4/data/utilities/utilities.meltano.yml#L5

pat_nadolny

08/15/2022, 2:30 PM

I think this should be part of the pip url when you install it via meltano so its a cleaner install. I'll open an issue to discuss it

vadiem_janssens

08/15/2022, 2:41 PM

That indeed looks like what should happen, I'm now able to run sqlfluff with dbt, but i'm getting some other errors. I've posted my question in #C01TCRBBJD7 as well, I'l create a thread with a thing i'm running into right now. Thank you for the help both!

Open in Slack

Previous Next