Hi everyone! What is the best way to add additional packages to Airflow (managed by meltano) installation? Ideally that would be

requirements.txt

file in

transform/

directory.

No transform is unrelated to airflow. Add it to the pip_url !

plugins:
  orchestrators: 
  - name: airflow
    pip_url: airflow[aws] aws-cli smart_open

This is kind of pseudo code cuz I just typed it on my phone but that's generally how you would do it. pip_url what you would put after pip install.

Yeah, I meant to write

orhestrate

Lmk if that works. Kind of curious.

Yes, it works

Fascinating approach of using

-r .../requirements.txt

along with other direct references in

pip_url

. Very cool - glad to hear this worked! 🔥

Agreed @aaronsteers 🙂

We had another use case come up recently where a pluralized "additional Pip installs" was found to be helpful, and if accepting a list of additional components, if could very likely also make sense to accept something like a requirements.txt as a list item in yaml, with or without the -r flag, since we could likely detect from a simple regex that it is a requirements file and not a library name.