Writing
Agents Need a Runtime, Not a Prompt Loop
An agent is no longer well described as a prompt inside a while loop. The useful abstraction owns execution around the model.
Notes
Last June, I wrote that the model was starting to behave less like a text generator and more like a control layer. Tool calling, code execution and structured outputs were turning inference into a loop: inspect state, choose an operation, execute it, observe the result, continue.
That loop is now becoming a software layer of its own.
OpenAI's March 11 release makes the shift unusually visible. The new Responses API combines model calls with built-in web search, file search and computer use. The Agents SDK adds handoffs, guardrails and tracing around single-agent and multi-agent workflows. OpenAI describes the problem directly: teams were spending large amounts of effort on prompt iteration and custom orchestration without enough visibility into what the agent was doing.
My takeaway is that an agent is no longer well described as a prompt inside a while(tool_call) loop. The useful abstraction is moving toward a runtime, or at least an agent harness, that owns execution around the model.
The prompt loop works until state starts to matter
The first version of an agent is easy to build: send the model a prompt, execute any requested tool, append the result and call the model again. It is the same basic function-calling loop many of us have been using since 2023.
The problems start when the task lasts longer than a few turns.
The system has to know which tool calls succeeded and which failed while returning plausible text, which intermediate results should stay in context, how much of the job is already done, whether a retry is permitted, when a specialist agent should take over, and how to inspect a run that failed yesterday when the same prompt works today.
None of that is solved by making the system prompt longer.
They belong to execution infrastructure.
OpenAI's Responses API is interesting for this reason. It is designed around multiple model turns and built-in tools rather than treating every request as one isolated completion. The API can combine web search, file search, computer use and developer-defined functions inside the same interaction. That moves some orchestration responsibility out of ad hoc application code and into a more explicit agent primitive.
A runtime owns more than the next token
I use "runtime" carefully here. The model itself is still not an operating system, and the SDK does not magically make an autonomous worker reliable.
The useful comparison is about responsibilities.
A runtime coordinates execution. It decides how operations are invoked, carries state between steps, exposes errors, applies boundaries and gives developers a way to inspect what happened.
The new Agents SDK is built around those concerns. OpenAI lists four core pieces: agents with instructions and tools, handoffs between agents, guardrails for input and output checks, and tracing for inspecting execution.
That is a different object from a prompt template. A prompt describes behaviour. A runtime has to manage behaviour after the model begins interacting with the world.
Anthropic reached a similar conclusion from another direction in its December guidance on agents. It separates predefined workflows from agents that dynamically choose tools and actions, while recommending simple, composable patterns over unnecessary framework complexity.
Tracing may matter more than another prompt technique
Once an agent can take several actions, observability becomes part of development.
With ordinary software, I would not debug a production pipeline by looking only at the final error message. I want logs, state transitions, timings and the inputs and outputs around the failing step.
Agents need the same treatment.
OpenAI is shipping tracing and observability as part of the Agents SDK rather than leaving it as an external add-on. That choice says something about the maturity of the problem. If an agent hands work to another agent, calls several tools and returns an incorrect result, developers need to reconstruct the execution path.
This is especially relevant because model behaviour is probabilistic. A conventional function usually fails in reproducible ways. An agent may choose a different route on the next run.
The trace becomes the equivalent of a production timeline: which model ran, what it saw, what tool it selected, what the tool returned, when control moved elsewhere and where the final state diverged from expectation.
Without that, prompt tuning turns into guesswork.
Tool execution needs boundaries
The second runtime responsibility is control.
Giving an agent more tools does not automatically make it more useful. It increases the action surface that has to be constrained.
The Agents SDK's guardrails are an early version of that boundary. A production system still needs deterministic checks outside the model: schema validation, permission checks, confirmation before destructive actions and verification after operations that change external state.
I would design a Blender or game-production agent the same way.
The model might decide that an asset needs to be re-exported. It should not invent the export path or silently overwrite the approved file. A tool can expose a typed operation with allowed presets and destination rules. After execution, another check can confirm that the expected artifact exists and matches the project constraints.
For a render pipeline, the agent can decide what needs investigation. The runtime should know which job IDs exist, which outputs were produced and whether a retry is safe.
For development work, the model can propose or apply a code change. The surrounding system should run tests, preserve the diff and keep enough execution history to explain what changed.
Models are good at interpreting intent and choosing among possibilities. Software remains better at enforcing invariants.
Multi-agent systems make orchestration visible
Handoffs are another signal that the abstraction is moving beyond one prompt loop.
OpenAI's SDK lets one agent transfer control to another. A triage agent can route a task to a specialist with different instructions and tools. The code example in the launch post is simple, but the architecture raises familiar distributed-system questions.
State has to cross the handoff in a defined shape, ownership of the task has to move with it, control may or may not be allowed to return, each role needs its own tool set, and the final result still has to be attributed and evaluated.
Not every application needs multiple agents; many will be easier to reason about with one model and a small tool set. But once specialisation becomes useful, orchestration stops being an implementation detail and becomes part of product behaviour.
What changes for production builders
For founders and developers, I would now separate three layers when designing an agentic product.
The model handles interpretation and decisions that benefit from learned reasoning. Tools expose controlled capabilities. The runtime manages state, execution, tracing, retries, handoffs and validation around them.
Keeping those layers separate makes model changes less disruptive. A better model can replace the reasoning component without forcing the application to redesign its permissions or execution logs. A new tool can be added without rewriting the entire prompt architecture. Evaluation can target the full run instead of one final message.
This resembles production pipelines I already trust in 3D and software work. Blender is not the asset pipeline. A renderer is not the render farm. A compiler is not the build system. Each powerful component sits inside infrastructure that manages inputs, execution state, errors and outputs.
Agents are heading toward the same separation.
My prediction
I think the next phase of agent development will be less about discovering one perfect prompt pattern and more about building better execution environments around models.
The model will remain the flexible part of the system. The surrounding runtime will become the place where reliability is engineered.
That runtime will probably stay fragmented for a while. Some teams will use vendor SDKs; others will build narrow internal loops. I expect the useful abstractions to converge around state, tools, permissions, handoffs, traces, evaluation and recovery.
The important change is conceptual.
An agent is not a chatbot that happens to call functions.
It is a program whose control flow is partly decided at inference time. Once that is true, it needs the same thing every non-trivial program eventually needs: a runtime around the part doing the execution.
Sources
- OpenAI, New tools for building agents, 11 March 2025.
- OpenAI, Responses API, announced 11 March 2025.
- OpenAI, Agents SDK, announced 11 March 2025.
- Anthropic, Building effective agents, 19 December 2024.
More
Other write-ups
Approval Is Not Publication
Thirty seven items in the queue, one approved, nothing published. The last arrow in the diagram is the only one that pays.
Neural Rendering Is Crossing From Reconstruction Into Synthesis
Reconstruction filled in what sparse sampling missed. DLSS 5 generates appearance the renderer never computed.
The Renderer Is Becoming a Training Data Engine
The renderer used to sit at the end of the pipeline. Physical AI gives the same scene a second job: teaching a model.
Local AI Is Becoming a Compute Fabric
Local AI meant one model on one machine. Routing inference across the devices already on a network changes the unit.
Agent Infrastructure Is Becoming a Product Category
Every team used to build the loop, the store, the sandbox. That layer is being sold rather than written.
Choosing a local model with a stopwatch, not a benchmark
Three models, five hard tasks, code actually executed. The official build was 2.4 times faster and more accurate than a community repack of the same model.
We measured real time lighting against baked light, and baked won
A 3D product simulation that had to look like an offline render. The real time version ran at 60 frames per second and looked like clay. Here is the measurement and the architecture that replaced it.
What actually broke in an agency run by agents
Three failures from a delivery stack that runs on AI. None of them were the model's fault, and all three reported success while producing nothing.
A Task Without A Check Command Is Not Automated
If a task has no command that can fail, the pipeline advances on the appearance of work.
A Gate The Model Writes Is A Gate The Model Loosens
Three quality gates returned green while the work behind them was wrong, each for a different reason.
The Scoring Model Was Wrong And It Put The Worst Lead First
A weighted sum let one axis substitute for the other, so a company with money and no problem ranked in the top twenty.
Building Software Got Easy. Getting Value Out Of It Did Not
Aristo took weeks to build. Everything after the build is still in progress, and that gap is the whole story.
Capability Is Becoming an Operational Risk Surface
Safety questions used to be about the text. Once a model can act, the capability itself becomes something to operate.
The Scene Graph Is Becoming an API for AI
A scene graph exists for artists and software. Agents are becoming another consumer, and they need structure rather than pixels.
Animation Is Moving From Clips to Motion Priors
Authored keyframes and blended clips are giving way to asking which constraints define acceptable motion.
Materials Are Becoming Learned Programs
A material is texture maps, parameters and shader code. It is starting to become a small learned program that answers a rendering question.
Procedural Systems Are Expanding Beyond Geometry
Geometry Nodes started with a narrow name. Blender 5.2 puts physics, sound and object data through the same graph.
The Unit of AI Work Is Becoming the Task, Not the Turn
Chat taught us to think one turn at a time. Long-running agents make the task the thing that is scheduled, resumed and reviewed.
Game Engines Are Becoming Operating Systems for Worlds
Engines have been judged on what they render and simulate. The Unreal 6 roadmap points at operating a world rather than drawing one.
The Model Is Becoming a Replaceable Backend
Choosing a provider used to mean choosing an architecture. A stable interface makes replacement possible and evaluation makes it safe.
The Harness Is Part of the Capability
The same model behaves differently depending on context policy, tool design and execution feedback. That surrounding software is not neutral.
Physics Engines Are Becoming Trainable Components
A simulator predicts what happens next. A differentiable one can answer which parameter should change to stop the failure.
The Agent Needs an Environment, Not Just Tools
A search function and a database query were enough for short loops. Longer work needs a place to stand.
Coding Agents Are Becoming General-Purpose Computer Workers
Repositories were a friendly environment: text in, terminal actions, checkable results. That was a starting point, not a boundary.
Open Standards Outlive Model Generations
A year after the MCP bet, the argument can be checked against what happened rather than what was hoped.
Colour Management Is a Pipeline Contract
Blender 5.0 reads as better display options. Giving a file an explicit working colour space is an architectural change.
The Best Model May Be a Router, Not a Model
GPT-5 moves model selection inside the system. The interesting unit stops being which model and becomes which compute policy.
World Models Are Not Game Engines Yet
Genie 3 generates a navigable 720p world at 24 fps. Production work needs state you can inspect when something goes wrong.
Memory Is Becoming a System Capability
A follow-up to the long-context argument. Storing, selecting and expiring facts is turning into a named part of the product.
Coding Agents Change the Unit of Software Work
AI coding tools have been judged where code appears on screen. The boundary moves when the agent owns a task instead of a snippet.
Agents Need Protocols Between Each Other, Not Just Tools
Tool calling solves the inside of the loop. It says nothing about one agent reaching another built by a different team on another platform.
Reasoning Is Not the Only Path to Better Models
Longer thinking improves maths and code. A model that solves a logic puzzle and misreads ordinary intent is not the better production model.
Rendering Is Becoming a Reconstruction Stack
Sparse samples, motion data and lower-resolution frames become a larger final result. Debugging becomes layered when reconstruction sits in the middle.
Agent Reliability Is an Evaluation Problem, Not a Prompting Problem
When an agent misses a step, the usual fix is a stricter prompt. The failure is more often in how completion is detected.
MCP Might Matter More Than Another Model Release
A model can reason well and still be useless inside a company if it cannot reach the files, repositories and tools where work lives.
Computer Use Is the Missing Layer Between Models and Software
Most integrations assume useful software exposes the right API. Much of real software never did.
Inference-Time Compute Is a New Scaling Axis
o1 improves when it is allowed to spend longer on a problem. A benchmark score without a compute budget is an incomplete number.
The Final Pixel Won't Come From the Renderer
Geometry, camera and scene structure stay reliable ground truth. More of final appearance is moving into learned systems.
Open Models Are Becoming Research Infrastructure
Llama 3.1 gets discussed as a benchmark result. The licence terms change which experiments are possible at all.
The Model Is Becoming a Runtime
Function calling, code execution and structured output turn inference into a loop. The model stops being a text generator and starts being a control layer.
Multimodality Changes the Architecture, Not Just the Interface
GPT-4o is easy to read as a faster interface. Training one model end to end across text, vision and audio is an architectural change.
Benchmark Scores Are Not Model Capability
Claude 3 posts 86.8% on MMLU and 50.4% on GPQA Diamond. The chart is useful and it is not the same thing as capability.
Long Context Is Not Memory
Gemini 1.5 makes a million tokens usable. A larger working set is not a system that decides what should survive the session.