https://linen.dev logo
Join Slack
Powered by
Is there a way to run a dbt transformer with tags ...
# getting-started
e
Is there a way to run a dbt transformer with tags to specify a dbt pipeline in the meltano run/elt commands. Something to the effect
Copy code
meltano run tap target (dbt:run --select tag:process1)
This idea works through invoke but I havent found a way to use tags for run/elt. Thanks
a
Hi, @eric_s. As of now, not with an inline notation. The recommended approach would be to add a named command which has the the additional args you require. Then, your above would look like:
Copy code
meltano run tap target dbt:run-postgres
Where "run-postgres" would be defined as a command in your meltano.yml, under the dbt plugin definition.
t
Note this only works for 
run
and not 
elt
e
Thank you both for the replies, I am new to meltano and figured I was missing something. I am attempting to implement the named command but I seem to be missing something else now. If I configure the meltano.yml to the style
Copy code
- name: dbt-postgres
  config:
      ...
  commands:
      run-process1: run --select tag:process1
I get the message
Copy code
Run invocation could not be completed as block failed: Cannot start plugin: Command 'run-process1' could not be found. Transformer 'dbt-postgres' supports the following commands: clean, compile, deps, run, seed, snapshot, test
I also tried using a custom plugin like
Copy code
transformers:
- name: dbt-postgres
  config: 
    ...
- name: dbt
  inherit_from: dbt-postgres
  commands:
    run-process1: run --select tag:process1
But then I get the error
Copy code
Error: Block dbt:run-process1 not found
c
FWIW, I never use 
--select
directly in a command, but only 
--selector
instead.
I am wondering if the 
tag:tagname
syntax is throwing off 
meltano invoke
...
e
I tested it and still no luck; as it says cmd not found and that calling from invoke works, I don't think it has to do anything with dbt. I appreciate the help though, I will probably transition to using selectors. Can I ask does the meltano.yml setup look correct?
t
@eric_s can you share more of your meltano.yml and the exact command you ran? Iā€™m wondering if you need to specify the 
executable
as 
dbt
.
alternatively you can try wrapping it in quotes as well:
Copy code
commands:
    run-process1: "run --select tag:process1"
the executable should already be set behind the scenes.
https://github.com/meltano/squared/blob/main/data/transform/transformers.meltano.yml is a more realistic example of how we use commands with dbt.
e
Thanks for sharing this example Taylor, I realized my mistake. I was trying to define the command in the environment not on the plugin; I initially had thought I was overriding the default command for the environment transformer and this train of thought led me the wrong direction.
t
šŸ™Œ glad you got it sorted šŸ˜„
a
Copy code
- name: dbt-postgres
  config:
      ...
  commands:
      run-process1:
        args: run --select tag:process1
I didn't see it at first, but on looking closer, I also realized the 
command
declaration should be an object with a subnode of 
args:
- similar to the above and the code sample from @taylor. Hope that helps!
Open in Slack
PreviousNext
Powered by