small-models

This is a follow-up to Small Models Decide, Code Does, LLMs Think. In that post I argued that the model is only a third of the answer: small models decide, code does, and LLMs think, only when they have to. It was about taking model calls out of a browser agent. This time I wanted the same split in the most common LLM interface there is: a chat box. Most chat messages don’t need a large language model....

I spent an evening taking model calls out of a browser agent. A Google Flights task went from 15 seconds to under 7. The biggest gains came from moving dates, ranking, and field values into code. That experiment changed how I think about where models belong in a system. What prompted it On September 15, TypeSafe AI launched Jev and called it a System One model, after Kahneman’s fast, intuitive System 1....