It's easy to default to the biggest, newest frontier model for every AI feature — it's the safest choice to defend in a meeting. But for a large share of production use cases, a smaller, cheaper, task-specific model performs just as well, at a fraction of the latency and cost, and it's worth understanding why before you commit to an architecture.
Where frontier models earn their cost
- Open-ended reasoning across unfamiliar domains, where you can't anticipate the range of inputs.
- Tasks requiring broad world knowledge the model wasn't specifically trained or fine-tuned on.
- Low-volume, high-stakes tasks where the cost difference is trivial compared to the value of getting it right.
Where small models are the better engineering choice
- High-volume, narrow tasks — classification, extraction, routing — where the input distribution is well understood.
- Latency-sensitive paths — anything in a real-time user interaction where a two-second round trip is the difference between a good and bad experience.
- Cost-sensitive scale — a task run millions of times a month where per-call cost compounds fast.
- Data residency or on-device requirements — where a smaller model can run locally or in a constrained environment a frontier API can't reach.
A simple decision framework
Ask three questions about the task:
- Is the input distribution narrow and well understood? (Favors small model.)
- Does latency or per-call cost matter at your expected volume? (Favors small model.)
- Does the task require broad reasoning outside a specific domain? (Favors frontier model.)
Two or more answers favoring "small model" is usually a strong signal to prototype with one before defaulting to a frontier API.
The frontier model is the right default when you don't yet know the shape of the problem. Once you do, it's often the wrong long-term choice for that specific task.
The takeaway
Treat model selection as an ongoing architecture decision per task, not a one-time company-wide choice. The best-performing production systems we've seen mix a frontier model for genuinely open-ended work with smaller, cheaper models doing the high-volume narrow work underneath it.