I spent an evening taking model calls out of a browser agent. A Google Flights task went from 15 seconds to under 7. The biggest gains came from moving dates, ranking, and field values into code.
That experiment changed how I think about where models belong in a system.
What prompted it
On September 15, TypeSafe AI launched Jev and called it a System One model, after Kahneman’s fast, intuitive System 1. It doesn’t write text. You give it state and a few questions, and it returns typed answers with probabilities in 70–500 ms.
It started an argument. Skeptics note that classification is decades old: “the problem is old; the architecture and product around it may be new.” Fans point at the speed and the price.
A common browser-agent pattern sends every step to an LLM: here’s the page, what next? Structured outputs make the replies easier to parse, but every step still pays for a large model. The question Jev raised for me wasn’t about output format. It was: which steps need model inference at all?
Three layers
| Layer | Good at | Fails by |
|---|---|---|
| Small model: decides | “Which of these?” Classify, route, pick, extract | picking the wrong option |
| Code: does | Anything you can compute: dates, math, sorting, checking | not handling a case it never saw |
| LLM: thinks | Open-ended text, new situations, planning | saying something plausible and wrong |
Each layer fails differently, and that’s the point. You put each job where its failure is cheapest to catch.
The fast layer needs a fence
In Kahneman’s book, System 1 is fast and error-prone. It jumps to conclusions.
The best critique of Jev lands exactly here: “can’t hallucinate” really means it can’t answer outside the options. It can still pick the wrong one.
So a fast model needs two fences:
- It only chooses from what exists. The options come from the real state: the buttons on the page, the categories in your schema. It can’t invent one.
- Code checks what it can. Not the model grading itself: plain code comparing the outcome to what was asked.
Be specific about what the check covers. In my agent, it confirms the search form holds every value from the goal, that the form was actually sent, and that the page isn’t an error or a captcha. It does not confirm that every result on the page matches. That verifies the submitted search, not the correctness of the results.
Fast guesses are fine when something cheap checks them.
Push work down the stack
My test bed was Zipline, a browser agent that runs entirely in Chrome on a small decision model (GLiNER2, on WebGPU). For the steps I could move down a layer, from LLM to small model or from small model to code, they got faster and more predictable:
- Dates moved to code. Once “the first Friday of next month” is parsed, calculating the date and selecting it on the calendar can happen in code. The calendar steps stopped calling the model, which removed 7.4 s of model time from a 15 s run.
- Ranking moved to code. “Cheapest red-eye” is a filter and a sort. Reading flight times with a pattern took the answer step from 1.7 s to 0.3 s.
- Field values moved off the LLM. With the optional small LLM switched off, the agent reuses the values the decision model already extracted from the goal. No extra model call per field, instead of up to 1.4 s each.
Numbers are single runs on my M3 Pro MacBook, models already loaded, one run per change. Read them as direction, not benchmarks.
“More predictable” has a concrete meaning here. One calendar step used to score a picker that was still closing, pick “Next”, and spend 2 s re-deciding. Code now waits for the picker to close and matches the day directly, removing both that timing failure and the model call.
What stayed with the model was the judgment call: is “Change ticket type” what “find a one-way ticket” means? Handwritten rules struggle to generalize that match, and an LLM is overkill for it.
What it costs
Speed is the visible win. Cost is the one that compounds.
A rough sum for an agent task of 10 steps, each sending about 5,000 tokens of page state and, for the LLM, getting about 200 tokens back:
| Where the step runs | Model API cost per task | Per million tasks |
|---|---|---|
| LLM ($0.20–$10 per million input tokens, output 5× that) | ~1.2–60¢ | ~$12,000–$600,000 |
| Hosted System One model (Jev: $0.042 per million input, output free) | ~0.21¢ | ~$2,100 |
| Small model in the browser | $0 | $0 |
| Code | $0 | $0 |
Prices are the published figures in TypeSafe’s launch post; the token counts are assumptions. These are illustrative API costs, not measured task costs or a comparison of equivalent task success.
Two things this table doesn’t show:
- A step moved entirely to code incurs no model API charge, whatever model prices do.
- “Free in the browser” means the cost moves, not disappears. The user’s device does the work, after a one-time download (614 MB for the model in my agent). For a free tool that’s a great trade. For a heavy workload it’s a design decision.
The honest part
Code is brittle. A site renames a field after you type in it, and a rule stops matching. Each surprise became a new rule, and an LLM would have taken some of them in stride.
The fix I’d want next is escalation. Confidence scores help only if they track actual correctness on the tasks you run. With that evidence, you can route uncertain decisions to an LLM or a person while keeping routine steps fast. Failed checks and pages the agent doesn’t recognize are escalation signals too. My agent has confidence floors today, not measured calibration, so this is the next experiment, not a result.
The shape it points to
Put together, the layers suggest a division of labor:
- LLMs help write the workflow ahead of time, and handle the exceptions at runtime.
- Small models make the routine calls inside it, repeatedly, with millisecond-scale inference.
- Code runs it, and checks what it can.
Less “an LLM in a loop”, more “an LLM that helps write the program, with small models as its if-statements.”
A rule of thumb
For any step, ask:
- Can explicit rules or calculations solve it? → code
- Does it require interpreting meaning within a bounded set of choices? → small model
- Does it require open-ended generation or planning? → LLM
In my agent, most steps landed on 1 or 2. That’s where the speed came from, and it’s why the whole thing fits in a browser tab.
Think slowly only when you have to.
Links
- Jev and System One models: TypeSafe AI’s launch post
- The debate: What everyone is getting wrong about Jev · systemonemodels.org
- Open models and runtimes: GLiNER2 · GLiNER2.5-Decide · open-jev by Nico Martin
- Agents built on the idea: Browser Use’s jev-ultrafast · gliner2-ultrafast
- What I built along the way: Zipline · Ad Spotter (ads found by GLiNER2.5-Decide)