Hi all, I am exploring Meltano and have a minimal POC running that extracts&loads, transforms with dbt and airflow is running the schedule - so far so awesome 😃 Now I would like to add a "second transformation" step, where I train a basic ML-model - or say run custom python code that uses the connections and schedule alraedy configured in Meltano. How would you approach this? What's sort of best practice? has anyone done this? Thanks a lot! Jonas

I haven't used this but anytime I think about running python with dbt my brain hops to https://github.com/fal-ai/fal

Cool, I'll see and let you know 🙂, thanks!

@jonas_krauss this has come up a few times recently. the current pattern would be to add your “second transformation” as a utility in meltano and then use the new

meltano run

command to trigger it after dbt. https://meltano.com/docs/command-line-interface.html#run This command is still in preview though, so not quite ready for full prime time.

Alright, sounds straightforward, thank you - Add as a utility following https://meltano.com/docs/plugin-management.html#custom-plugins - right?

Interested to hear how this all panned out, we are attempting something similar in the near future

+1