Frontier models get the headlines. Small models get the traffic. For most production workloads, that is exactly the right division.
TL;DR
- Most production LLM traffic is routine — classification, extraction, formatting, short summaries.
- Small and open-weight models handle that work at a fraction of the cost and latency.
- Reserve frontier models for genuine reasoning; let small models do the volume.
The quiet inversion
Public attention tracks the frontier: the biggest model, the highest benchmark, the newest release. But look at what production systems actually do all day and the picture inverts. The bulk of requests are not hard. They are: classify this ticket, extract these fields, summarize this paragraph, decide which bucket this belongs in. For that work, a frontier model is overkill — slower, pricier, and no more correct.
Open-weight models have made this inversion economical. A DeepSeek-class mixture-of-experts model rivaling previous-generation frontier quality at a fraction of the price-per-token turns "use a small model for the routine stuff" from a compromise into the obvious default.
Why smaller wins the volume
- Cost. At high volume, the price gap between a small model and a frontier one is the difference between a viable product and a money pit.
- Latency. Small models respond faster — decisive for anything a user is waiting on.
- Control. Open-weight models can run on your own infrastructure, which matters for privacy, compliance, and predictable cost.
- Good enough, verifiably. For bounded tasks, a small model with a tight prompt and an eval often matches a big one where it counts.
Capability you do not need is just latency and cost you are paying for. Match the model to the task.
Where small models shine
- Classification and routing — intent detection, triage, tagging.
- Extraction — pulling structured fields from semi-structured text.
- Formatting and transformation — reshaping data, rewriting to a template.
- First-pass drafting — generate cheaply, escalate only the hard cases.
Where you still want the frontier
Small models are not magic. Keep the big model for:
- Deep reasoning — architecture, ambiguous trade-offs, multi-step planning.
- High-stakes correctness — where a wrong answer is expensive.
- The escalation path — when a small model's output fails a confidence or validation check.
This is just model routing in practice: the small model does the many easy jobs, the frontier model does the few hard ones.
The takeaway
The future is not one giant model doing everything. It is a fleet — small and cheap for the routine majority, large and capable for the demanding minority — orchestrated deliberately. The teams that internalize this run faster and cheaper than the ones still routing every request to the most expensive model they can find.