We just published a piece to the blog that blew me away: @pat_nadolny just did a deep dive on building LLM applications, and why Meltano (and other data eng tools) can be used to make it a lot easier. I just tried it out myself (adapted it to chat with my own blog) and it works like a charm, is super quick to modify, and really neat to use. For all the details, read this: https://meltano.com/blog/llm-apps-are-mostly-data-pipelines/

for context on Langchain… it was started before the GPT3.5 / chatGPT API was released, so at the time it was better then using OpenAI directly for many use cases. There was also a bit of FUD around getting “locked in” to OpenAI and if you used Langchain you’d be able to easily switch to another model provider. How it’s actually played out / my take: You’re probably better off just using OpenAI directly. GPT4 is better then any other model for most use cases and it’s easy enough to hit the API without adding the overhead of a lib. Use 3.5 if you need speed / a cheaper model. You also get Function Calling which can be really useful and I’m not sure how well langchain supports it. If / when the open source models or Anthropic start to catch up, Langchain will be a nice abstraction option, but it’s currently not worth the overhead in my opinion.

Awesome post! Would love to see more about the checkpointing, and techniques for loading only data that has changed in the source to avoid recalculating embeddings for $$$ purposes. To some degree this is what I am attempting to build internally using Meltano 🙂

@ben_theunissen maybe Pat will do a second iteration on it to incorporate that, I know he's been thinking about it already.

If you ever need someone to kick ideas around in the area of using Meltano for AI applications I am happy to discuss, going to be doing a lot more in the area over the next 3-6 months