Just onboarding to Dagster, trying to orchestrate Meltano, dbt and GoodData with it.
I executed all dbt models successfully, I can see each dbt model as asset in Dagster and can display its lineage.
I successfully executed Meltano as a job, I followed the Dagster tutorial(
https://dagster.io/blog/dagster-meltano-integration-tutorial) and dagster-meltano README(
https://github.com/quantile-development/dagster-meltano).
Now I am thinking about how to connect these two world together in Dagster.
My idea is to load Meltano jobs as assets and define dependencies between Meltano and dbt assets.
For instance, I extract/load github data from repo and org levels (two different taps). I would like to define two dependencies:
• tap-github-repo -> dbt models commits, pull_requests, ...
• tap-github-org -> dbt model repositories
The example in the dagster-meltano repo shows how to expose meltano jobs into Dagster. But I would like to expose assets so I can include them into the Dagster lineage.
Anyone tried to do something like that?