Notes on LLMs

Like most of the tech universe, I’ve been learning about LLMs. Here are my notes and observations.

Macro

• Unlike past eras, the major tech players are reacting quickly and leading the way on integrating AI. This makes it much more challenging for startups to find opportunities.
• Mistral AI strategic memo
• Coatue on macro market Q2 ‘23

Background Reading

Because LLMs are such a black box, you really want to set up an MLOps workflow, and then continuously experiment and tune your LLMs.

• https://vickiboykis.com/what_are_embeddings/
• https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning

Information Retrieval

I’m particularly interested in NLP to SQL.

• Text to SQL Leaderboard: https://yale-lily.github.io/spider
• How Canvas integrated it: https://canvasapp.com/blog/text-to-sql-in-production
• LangChain model: https://blog.langchain.dev/llms-and-sql/

Developer Libraries

• LangChain is a popular framework. I tried using it. It’s not clear to me that it really saves time in production. Feels like it’s really about exploratory code.
• I’m curious about LlamaIndex, which sort of competes with LangChain.
• HuggingFace has a broad set of libraries that help with training & managing your own LLMs.