The Best Model May Be a Router, Not a Model

For most of the LLM era, model selection has been a user or developer decision. Pick the fast model for ordinary work. Pick the reasoning model when the task is harder. Pick a smaller model when cost matters.

GPT-5 changes that interaction in ChatGPT. OpenAI describes it as a unified system made from a fast model, a deeper reasoning model, and a real-time router that decides which one should handle the request. The router looks at conversation type, complexity, tool needs and explicit user intent.

That architecture interests me more than the name GPT-5.

The strongest AI product may not be one model that handles every request the same way. It may be a system that decides how much intelligence, latency and compute a task deserves before choosing how to solve it.

Model selection is becoming a runtime decision

The model picker made sense when the difference between models was easy to understand.

GPT-4o was fast and general. The o-series spent more inference compute on hard reasoning. A user could choose based on the task.

But that choice requires the user to diagnose the problem before the system solves it.

OpenAI is moving that decision inward. In ChatGPT, GPT-5 routes ordinary requests to a faster model and harder ones to GPT-5 Thinking. OpenAI says the router can use signals including task complexity, tool requirements and instructions such as asking the system to "think hard." The company says the router is trained from signals such as user model switches, preference data and measured correctness.

The implication is simple: selecting the computational strategy is becoming part of inference itself.

That is a different problem from making one checkpoint larger.

Compute should follow task difficulty

Last year, o1 made inference-time compute visible as a scaling axis. A harder problem could receive more computation after the prompt arrived instead of depending only on a larger pretrained model.

GPT-5 turns that idea into product architecture.

Not every request benefits equally from extra reasoning. OpenAI's developer guidance says GPT-5 supports minimal, low, medium and high reasoning effort. It specifically notes that increasing reasoning beyond low adds little to some simple long-context retrieval tasks, while harder visual reasoning tasks gain more.

That is exactly the trade I would expect production systems to make.

If I ask an assistant to rename a group of exported files according to an existing rule, I do not need maximum reasoning. If I ask it to diagnose a rendering failure that crosses Blender scripts, asset state, render settings and external tools, spending more compute may be justified.

The useful unit is no longer "which model do I use?"

It becomes "what compute policy does this task need?"

Routing can include more than reasoning depth

The GPT-5 implementation makes routing visible mainly as a choice between fast responses and deeper reasoning, but I think the broader architecture extends further.

A production system can route across several dimensions:

• model size;
• reasoning budget;
• tool access;
• latency target;
• cost ceiling;
• local versus hosted execution;
• specialist versus general model.

The decision does not have to happen once.

A task could begin cheaply, discover that it is harder than expected, then escalate. A fast model could classify or prepare the task before a stronger model handles the difficult section. A reasoning model could be reserved for the small part of the workflow where additional inference compute changes the result.

This resembles ordinary compute scheduling more than traditional chatbot design.

We already do this elsewhere in software. Render farms assign different hardware or quality settings depending on the job. Build systems skip work that does not need to be repeated. Game engines allocate expensive effects selectively rather than running the maximum-quality path for every pixel.

AI inference is starting to need the same discipline.

One unified product can hide several computational systems

There is an important distinction between GPT-5 in ChatGPT and GPT-5 in the API.

OpenAI says the ChatGPT product is the routed system: fast model, reasoning model and router. In the API, gpt-5 refers to the reasoning model used for maximum performance, with separate gpt-5-mini and gpt-5-nano variants and configurable reasoning effort. The non-reasoning model used by ChatGPT is exposed separately as gpt-5-chat-latest.

That distinction matters because "GPT-5" is no longer one simple computational object across every surface.

From the user's perspective, a single product name can represent several execution paths.

I suspect this becomes normal.

People should not need a mental map of model families to ask a question. The application should know enough about cost, latency and task difficulty to choose a sensible path, while still exposing controls when the user or developer needs them.

Routing creates its own failure mode

Automatic routing removes complexity from the user, but it does not remove the decision.

It moves the decision into infrastructure.

If the router underestimates a task, the system can answer quickly and badly. If it overestimates everything, latency and cost rise without useful quality gains. Two superficially similar prompts may take different paths, which can make behaviour harder to reproduce.

For developers, that means the router itself needs evaluation.

I would want to know how often simple tasks are escalated unnecessarily, how often difficult tasks remain on the fast path, and whether routing changes tool-use reliability. Evaluation should measure the complete system, not only each underlying model in isolation.

This connects directly to the agent reliability problem. Once a system contains models, tools, memory, runtimes and routers, "model quality" explains only part of the observed behaviour.

The orchestration policy becomes part of capability.

Production systems should optimise a frontier, not a score

The usual benchmark question is: which model scores highest?

A routed system has a different optimisation target.

For each task distribution, I care about the relationship between quality, latency and cost. There may be no single best operating point.

A product-facing assistant may prefer a fast answer for most requests and escalate only when confidence is low. A coding agent modifying a repository may accept slower responses because a bad change costs more than additional inference. A real-time creative tool may have a strict latency budget and use deeper reasoning only for background operations.

GPT-5's API design reflects part of this idea. Alongside three model sizes, developers can vary reasoning effort and output verbosity. OpenAI explicitly recommends experimenting with the trade between performance, cost and latency for the actual use case rather than assuming the maximum setting is always best.

That is how I would design model infrastructure now.

Not "use the smartest model everywhere."

Use the cheapest execution path that still meets the quality requirement, then escalate when the task gives you evidence that it deserves more.

My prediction

I think model routing will become a larger part of AI system design than model selection.

Users will increasingly interact with one logical assistant while the infrastructure underneath chooses between several computational paths. Developers may still pin models when reproducibility or compliance requires it, but general-purpose products will benefit from dynamic allocation.

The interesting systems will not only route between model names. They will route between levels of reasoning, specialists, tools and execution environments.

Over time, this could make the question "which model are you using?" less informative.

A better question may be: what policy decides how this task gets solved?

GPT-5 is still a model family, and the router does not replace the intelligence of the models underneath it. A weak model does not become strong because a router selected it correctly.

But once several strong models offer different cost, speed and reasoning profiles, choosing among them becomes its own source of system performance.

The best model may still matter.

The best product may be the one that knows when not to use it.

Sources

1. OpenAI, Introducing GPT-5, 7 August 2025.
2. OpenAI, Introducing GPT-5 for developers, 7 August 2025.
3. OpenAI, GPT-5 System Card, 7 August 2025.