Hi all, I am working on a meltano project that int...
# troubleshooting
m
Hi all, I am working on a meltano project that integrates data from multiple sources and uses PostgreSQL as target. I am using DBT-Postgres for transformations, and I was wondering if I can run transformations for a single model. I do not want to transform all my data every time. Is there a way? I have seen the original DBT project uses the
--select
flag to specify this, but couldn't make it work with meltano. Thank you! https://docs.getdbt.com/faqs/runs/run-one-model
t
yep! just need to define a command if you want to run it as part of
meltano run
. otherwise you can use
meltano invoke
and specify any command line arguments you normally would
u
Heres an example of how I define my dbt command in my meltano project if its helpful https://github.com/meltano/squared/blob/ce939c6bf15a571ce2b23e4384a4b0774d07b099/data/transform/transformers.meltano.yml#L15. Its a little more work to add a new named command each time but it actually scales way better because as the project grows my select criteria is getting more and more complex so its nice to have it in code.
a
Can we just rename
squared
to
advanced_meltano
please!? Awesome resource.
m
Great info! Thanks peeps!
m
Its a little more work to add a new named command each time but it actually scales way better because as the project grows my select criteria is getting more and more complex so its nice to have it in code.
Not to say that you shouldn’t do this in meltano, but it’s possible to define them in YAML with dbt also as YAML selectors https://docs.getdbt.com/reference/node-selection/yaml-selectors