Hi Melty Crew, I wanted to set up a toy example o...
# getting-started
d
Hi Melty Crew, I wanted to set up a toy example of a dbt-meltano-dagster combo. I used this awesome starting guide: https://medium.com/@kenokumura/how-to-orchestrate-dbt-with-dagster-in-multi-containers-on-docker-ebf0d171a3a9 and now added meltano and dagster-meltano to the Dockerfile_user_code, mounted my meltano project, and added a "load_jobs_from_meltano_project" to repo.py. Doing only dagster+meltano or dagster+dbt this way works, but together it breaks (something wrong with snowplow-tracker...), which I suspect is a version thing. To get started, which version of these packages should be compatible here? I trial-errored some combinations but got nowhere. Thanks a lot!!
e
Hey @Dries Beheydt 👋 The latest stable versions (Meltano 3.7, dbt 1.9) should be compatible. If they're not, let me know cause that'd be a regression.
a
Here is my definitions:
Copy code
utilities:
  - name: dagster
    variant: quantile-development
    pip_url: git+<https://github.com/quantile-development/dagster-ext@v0.1.3> dagster==1.9.13 dagster-postgres
      dagster-dbt dbt-postgres<1.8.0 dagster-azure dagster_msteams elementary-data dagster_powerbi
ty 2
And to start dagster it's
meltano invoke dagster:dev
with this definition.
Copy code
dev:
        args: dev --workspace $REPOSITORY_DIR/workspace.yml --dagit-host 0.0.0.0
        executable: dagster_invoker
I'm sure it's not super great to be running with
dev
in production but I never got my head around the multi-container setup.
Happily using dagster + meltano+dbt in production for over a year now so happy to assist. You definitely need to stay at dagster 1.9 as there was a big change in dag 1.10 which has not been resolved yet in the dagster-meltano integration.
d
Yes! starting from dagster==1.9.13 indeed worked out! Thanks so much!!
🙌 2