All posts

Evaluating LLMs Like an Engineer: Benchmarks vs. Reality

AIMachine LearningBest PracticesSoftwareDevelopment

A model topping a leaderboard tells you it is good at the leaderboard. Whether it is good at your problem is a different question — and only your eval answers it.

TL;DR

  • Public benchmarks measure general capability, not fitness for your task.
  • Know what each benchmark actually tests before quoting it.
  • Build a small, task-specific eval set — it beats any leaderboard for decisions.
  • Track quality, cost, and latency together; a model only wins on all three for a reason.

What the popular benchmarks actually measure

Numbers fly around without context. A quick decoder for the ones you will see most:

  • GPQA Diamond — graduate-level science questions. Tests deep reasoning, not coding or tool use.
  • OSWorld — completing real tasks in a desktop environment. A proxy for computer-use and agentic capability.
  • SWE-bench — resolving real GitHub issues. The closest public proxy to "can it do software engineering," but still not your codebase.
  • EQ-Bench — emotional and social reasoning. Relevant for support and conversational products, not for compilers.

A model scoring 93% on GPQA might still be mediocre at your specific extraction task. Capability is not transferable in the way a single headline number implies.

A benchmark is a standardized test. Your application is the real exam, and the questions are different.

Why leaderboards mislead

Three reasons to treat them as a starting filter, not a verdict:

  • Contamination. Popular benchmarks leak into training data over time, inflating scores.
  • Distribution mismatch. Benchmark tasks rarely look like your prompts, your domain, or your formats.
  • Single-axis framing. "Best model" usually ignores cost and latency, which decide real-world viability.

Build the eval you actually need

You do not need an ML platform to evaluate well. You need a representative set of real inputs and a way to judge outputs:

  1. Collect 20–50 real examples from your domain, including the messy edge cases.
  2. Define what "good" means per example — exact match, a rubric, or an LLM-as-judge with a clear scoring guide.
  3. Run candidate models against the set and compare on quality, cost, and latency.
  4. Re-run it on every model upgrade. A "better" new model can quietly regress on your specific task.
For each candidate model:
  score   = average(quality over your 50 examples)
  cost    = tokens × price
  latency = p95 response time
Pick the cheapest model that clears your quality bar — not the top of the leaderboard.

The engineering mindset

Treat model selection like any other dependency choice: measure it against your requirements, not its marketing. A frontier model that costs 10× and is 2% better on your eval is rarely the right call when a smaller model already clears the bar.

The teams who win the model-release season are not the ones who always run the newest, biggest model. They are the ones who can answer, with data, "is this actually better for us?"

More on building with AI on the blog. →