The question is no longer "which model is best?" It is "which model is right for this task, at this cost, this week?"
TL;DR
- Model choice is now a per-task decision, not a one-time bet.
- Classify tasks by reasoning depth, latency tolerance, and cost sensitivity.
- Use frontier models where judgment matters; route routine work to cheaper, smaller ones.
- Re-evaluate on a cadence — the frontier moves every few weeks now.
The new reality
Late 2025 saw a frontier model land roughly every week. That pace makes "we standardized on one model" a fragile strategy. The teams getting leverage treat models like a portfolio: different tools for different jobs, swapped as the landscape shifts.
A framework: classify the task first
Before picking a model, characterize the work along three axes:
- Reasoning depth. Multi-step architecture and ambiguous trade-offs need a frontier model. Reformatting JSON does not.
- Latency tolerance. A user waiting on a UI needs speed; an overnight batch job does not.
- Cost sensitivity. A call made millions of times a day has very different economics from one made on demand.
High reasoning + low volume ─▶ frontier model (architecture, hard debugging)
Low reasoning + high volume ─▶ small/cheap model (classification, formatting)
High latency tolerance ─▶ bigger model, batch it
User-facing, latency-critical ─▶ fastest model that clears the quality bar
Using a frontier model for everything is like hiring a principal engineer to reset passwords. It works, but you are lighting money on fire.
Match, don't max
Concrete mappings that hold up well:
- Architecture, ambiguous bugs, security-sensitive logic → the strongest model you have. The cost of a wrong answer dwarfs the token cost.
- Unit tests, docstrings, commit messages, summaries → a smaller, cheaper model. These are high-volume and well-bounded.
- Classification, extraction, routing → a small model with a tight prompt and an eval to keep it honest.
Make switching cheap
Because the frontier keeps moving, design so that changing models is a config change, not a rewrite:
- Abstract the model behind a thin interface.
- Keep prompts and the model id in configuration.
- Maintain a task-specific eval (see evaluating LLMs like an engineer) so you can re-test new models in minutes.
The discipline that pays off
Re-run your evals when a notable model drops, but do not reflexively upgrade. "Newer" and "bigger" are not "better for your task." The framework keeps you anchored: classify the work, match the model, verify with your own data, and keep the switching cost low. Do that, and each release becomes an opportunity rather than a fire drill.