Hello, I'm just getting started with meltano. I do...
# getting-started
m
Hello, I'm just getting started with meltano. I don't want to move all tables to target only some of them is needed, (I learned about the
meltano select
filter). We already have an existing dbt project. I wonder if it's possible to set the select configuration (tables to be moved from source to target) dynamically based on the source.yml files in the dbt project, or do I have to type them one by one?
v
Possible, but not easy. You have to write a script of some kind. I've wrote a few myself, Pat has one that is public somewhere with Airflow. But they all get tweaked pretty heavily based on your use case Easiest approach for me has been to use environment variables in meltano to select the stream you want, and correlate that with your related dbt model. Since we tend to use only a few sources this normally aligns pretty well I really only do this when it's greater than 50 source tables. But context matters 🤷