Meltano

image.png

Does anyone have a successful integration with dagster and meltano?

I am trying to figure out how they both integrate together, how to load assets automatically etc.

Yes. I use Jules' dagster_meltano module and his example dagster repository that are available inside the `dagster-ext` meltano extension. Works great for me.

Thanks <@U06D0HF662U> let me check it out, might have more questions for you.

So you dont use <https://hub.meltano.com/utilities/dagster/> ?

That's the code that I use. I just don't run it from "within meltano" though. i.e. I don't use the "wrapper" commands like `meltano invoke dagster:initialize` or `meltano invoke dagster:run`

Instead I use a more common supervisor (`systemd`) to run the dagster daemon and the dagster web ui (dagit) but I still use the same dagster meltano integration modules under the hood

Let me know if you have any issues getting it all set up.

Perhaps you have an example of a repository how you use it? It confuses me that obsolete arguments are used here. I don't quite understand how dagster sees all my taps and targets.

Or should I describe the dagster in each of my taps, and then launch it somehow

&gt; It confuses me that obsolete arguments are used here.
What are the arguments that are obsolete?

&gt; should I describe the dagster in each of my taps, and then launch it somehow
I don't think that is necessary. The way I use the dagster meltano module is in its simplest form. All I use if for is to be able to:
1. See all job definitions in dagster
2. Launch jobs on demand in dagster
3. View the status of scheduled  job runs in dagster

Exactly what I want at this stage.
The only thing I still need is to run 1000 and. more taps with different authorization data (with different environments)

thought folks in this thread would be interested in this: <https://dagster.io/blog/dagster-meltano-integration-tutorial>

(But it sounds like you already have the integration up and running… and this is more of an intro tutorial!)