is anyone using either airflow or dagster to orchestrate meltano and have opinions on how it's going? I'm going through an orchestrator / elt process re-evaluation right now and would appreciate any war stories

we’re just getting started using dagster to orchestrate meltano and its been a mostly smooth experience so far barring one small hiccup

We're in the middle of setting something up too and it's been smooth, but it's all been local dev so far because we're still setting up a production Dagster deployment. The fun part is that you can build a deeper integration that you might be able to easily with Airflow - for example we hooked Meltano environments and dbt targets up to Dagster job config so that running a "dev" job instead of "prod" just means setting one job level config option.

thats rad @prratek_ramchandani - agreed that there is some really slick stuff you can wire up with Dagster. We’re also experimenting with yielding AssetObservations from our meltano runs based on the METRIC messages emitted from extractors

I love that! It could be fun to collaborate on a dagster meltano plugin at some point. We parsed messages from the Bigquery target to yield asset materializations with row counts, load times, bytes, etc.

thanks team, this is helpful. i am definitely way more excited about the prospect of using dagster than other tools, glad to hear it works well. will need to look into asset observations