General Question on transformations vs models.. I have some tables which.. I want to process with what most call rule-based models.. I could put those into a flow outside of meltano.. but it feels like I'd need to then configure more "stuff"... I am a newb with apache airflow but.. is the right way forward to... write my rule-based models in JuliaLang / Python / etc... and schedule those separately from meltano using airflow? If I start to use machine learning I will be likely using MLFlow.. where again I can use some apache airflow type setup to register models and execute them against a dataset/sets... any thoughts here? as I understood it, all current transformations are done in DBT.. which is more the dataops side and supports only SQL .. but as you move into ML and UDFs.. those would I guess fall outside of meltano? Thank you

Those would currently fall outside Meltano, yes, but if it is pip-installable you can use a utility to manage configuration for it still

thanks @taylor, is it recommended I imagine to run a separate instance of airflow or.. try to reuse the one packaged with meltano? I'm guessing the latter, if possible, allows for better logic around results / model output table creation(s)

Running Airflow through Meltano does create a separate instance of airflow - you can throw any DAG you want into it