Writing
The Agent Needs an Environment, Not Just Tools
A search function and a database query were enough for short loops. Longer work needs a place to stand.
Notes
Tool calling gave language models a way to reach outside the prompt. Give the model a search function, a database query, or an API, and it can choose when to use it.
That was enough for short agent loops. It is not enough for the kind of work agents are starting to do now.
OpenAI's March 11 Responses API update makes the missing layer visible. The model can work with a shell tool inside a hosted container that has a filesystem, optional structured storage such as SQLite, restricted network access, reusable skills and native context compaction. The model still does not execute commands by itself. It proposes actions; the surrounding system runs them and feeds the results back.
My takeaway is that an agent needs more than tools. It needs somewhere for work to exist while the task is in progress.
A tool is an operation. An environment is state.
A function call is usually narrow. Search for a file. Query a table. Send an HTTP request. Run a command.
A real task connects many of those operations through intermediate state.
Suppose an agent has to collect sales data from an API, compare it with last quarter, produce charts, write a short analysis and deliver a spreadsheet. The API response is not the final result. The agent needs somewhere to store raw data, transformed data, scripts, temporary charts and the workbook it is building.
Trying to keep all of that inside conversation context is a bad fit.
OpenAI's new environment treats the container as working context. Files can be staged in a filesystem. Structured data can live in SQLite. Shell processes can transform that data. The result can remain as an artifact instead of being serialised back into the prompt after every step.
That is closer to how ordinary software works. Programs do not keep their entire state inside one function argument. They use filesystems, databases, processes and services around them.
Agents are reaching the same point.
The context window should not become a filesystem
Long context made it tempting to solve every state problem by putting more information into the prompt.
That works until the task starts producing its own data.
A shell command may return thousands of lines. A CSV can contain millions of values. A research task may download documents that only matter for one calculation. A code task can generate build logs, test results and intermediate patches.
The model does not need all of those bytes occupying attention at the same time.
OpenAI recommends staging large inputs in the container and letting the model inspect only the files or rows it needs. For structured data, the system can expose the table schema and let the model query relevant records instead of copying the entire dataset into context. Shell output can be capped so raw logs do not consume the working window.
This is the same separation I argued for with memory: active context is working memory, not durable storage.
The environment gives the agent another level in that hierarchy.
Long-running work needs continuity outside the prompt
Once a task lasts long enough, even a carefully managed context window fills up.
OpenAI's answer is compaction. The Responses API can compress prior conversation and tool state into a smaller representation, then continue the workflow across context boundaries. OpenAI says Codex uses the same mechanism for long-running coding tasks.
The architectural point matters more to me than the exact compaction method.
A long task cannot depend on every prior token remaining directly visible forever. The system needs ways to preserve what still matters while leaving low-value history behind.
This makes agent state look less like chat history and more like process state.
The current prompt contains what the model needs to decide next. The filesystem contains working artifacts. A database can hold structured state. The runtime retains execution state. Compaction carries forward a reduced history of the reasoning loop.
No single storage layer has to do everything.
Network access is part of the environment too
Giving an agent a shell without network rules would create another problem.
Many useful workflows need external access. The agent may need to call an API, download a package, query a service or retrieve current data. But unrestricted outbound access means the same environment can send information somewhere it should not go.
OpenAI's hosted containers route outbound requests through a policy layer with allowlists and access controls. Credentials can be injected only for approved domains, while raw secrets remain outside model-visible context.
That is a useful design pattern beyond one API.
Permissions should belong to the environment, not to the model's judgment.
The model can decide that it wants to contact a service. Infrastructure should decide whether that destination is allowed and whether the required credential can be used there.
This becomes more relevant as agents move from code repositories into business systems. The broader the task, the more dangerous an unrestricted execution environment becomes.
Coding agents showed why this architecture works
Coding agents reached this problem earlier because repository work already requires an environment.
The Codex app gives parallel agents isolated worktrees so they can change the same repository without touching one another's Git state. It uses sandboxing, controlled network permissions and reviewable diffs around long-running work. OpenAI describes Codex as moving from writing code toward using code to complete broader work on a computer.
Research before these products pointed in the same direction. SWE-agent found that changing the interface between the model and the computer could materially change software-engineering performance. Its argument was simple: agents, like human users, perform differently depending on the environment and interface they are given.
That makes "model capability" an incomplete description of an agent system.
A capable model with a weak environment may spend time rediscovering files, lose intermediate state or receive poor feedback from commands. The same model inside a better execution setup can behave like a much stronger worker because the surrounding system makes state legible and actions verifiable.
What this means for production pipelines
This maps directly to the kind of Blender, rendering and game-development automation I care about.
Imagine an agent preparing a batch of product scenes for delivery. It may need to inspect project files, read naming rules, run Blender scripts, collect validation results, generate preview renders and prepare a report of failures.
I would not want every intermediate result pushed back into a chat transcript.
The agent should have a task workspace. Input files can be mounted or copied into it. Scripts can run there. Validation results can be written to structured files. Render previews can remain as artifacts. A final report can link back to those outputs.
The same applies to game-development work. A build investigation may involve source files, compiler output, logs, test artifacts and generated patches. The environment becomes the place where the investigation exists while the model decides what to do next.
This separation makes debugging easier too. If the agent fails, I can inspect the files and state it left behind rather than reconstructing the whole task from natural-language messages.
My prediction
I think the environment will become a first-class part of agent architecture.
Tool catalogues will still matter, but they describe what an agent can request. The environment determines where those requests execute, where intermediate state lives, what resources can be reached and what survives between steps.
As agents take on longer jobs, I expect products to expose this layer more directly: isolated workspaces, durable artifacts, structured task state, scoped network access, reusable skills and policies around execution.
That will make the model only one part of the capability stack.
The model decides what should happen next.
The environment is where the work actually happens.
Sources
- OpenAI, From model to agent: Equipping the Responses API with a computer environment, 11 March 2026.
- OpenAI, Introducing the Codex app, 2 February 2026.
- OpenAI, Introducing GPT-5.3-Codex, 5 February 2026.
- Yang et al., SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering, 2024.
More
Other write-ups
Approval Is Not Publication
Thirty seven items in the queue, one approved, nothing published. The last arrow in the diagram is the only one that pays.
Neural Rendering Is Crossing From Reconstruction Into Synthesis
Reconstruction filled in what sparse sampling missed. DLSS 5 generates appearance the renderer never computed.
The Renderer Is Becoming a Training Data Engine
The renderer used to sit at the end of the pipeline. Physical AI gives the same scene a second job: teaching a model.
Local AI Is Becoming a Compute Fabric
Local AI meant one model on one machine. Routing inference across the devices already on a network changes the unit.
Agent Infrastructure Is Becoming a Product Category
Every team used to build the loop, the store, the sandbox. That layer is being sold rather than written.
Choosing a local model with a stopwatch, not a benchmark
Three models, five hard tasks, code actually executed. The official build was 2.4 times faster and more accurate than a community repack of the same model.
We measured real time lighting against baked light, and baked won
A 3D product simulation that had to look like an offline render. The real time version ran at 60 frames per second and looked like clay. Here is the measurement and the architecture that replaced it.
What actually broke in an agency run by agents
Three failures from a delivery stack that runs on AI. None of them were the model's fault, and all three reported success while producing nothing.
A Task Without A Check Command Is Not Automated
If a task has no command that can fail, the pipeline advances on the appearance of work.
A Gate The Model Writes Is A Gate The Model Loosens
Three quality gates returned green while the work behind them was wrong, each for a different reason.
The Scoring Model Was Wrong And It Put The Worst Lead First
A weighted sum let one axis substitute for the other, so a company with money and no problem ranked in the top twenty.
Building Software Got Easy. Getting Value Out Of It Did Not
Aristo took weeks to build. Everything after the build is still in progress, and that gap is the whole story.
Capability Is Becoming an Operational Risk Surface
Safety questions used to be about the text. Once a model can act, the capability itself becomes something to operate.
The Scene Graph Is Becoming an API for AI
A scene graph exists for artists and software. Agents are becoming another consumer, and they need structure rather than pixels.
Animation Is Moving From Clips to Motion Priors
Authored keyframes and blended clips are giving way to asking which constraints define acceptable motion.
Materials Are Becoming Learned Programs
A material is texture maps, parameters and shader code. It is starting to become a small learned program that answers a rendering question.
Procedural Systems Are Expanding Beyond Geometry
Geometry Nodes started with a narrow name. Blender 5.2 puts physics, sound and object data through the same graph.
The Unit of AI Work Is Becoming the Task, Not the Turn
Chat taught us to think one turn at a time. Long-running agents make the task the thing that is scheduled, resumed and reviewed.
Game Engines Are Becoming Operating Systems for Worlds
Engines have been judged on what they render and simulate. The Unreal 6 roadmap points at operating a world rather than drawing one.
The Model Is Becoming a Replaceable Backend
Choosing a provider used to mean choosing an architecture. A stable interface makes replacement possible and evaluation makes it safe.
The Harness Is Part of the Capability
The same model behaves differently depending on context policy, tool design and execution feedback. That surrounding software is not neutral.
Physics Engines Are Becoming Trainable Components
A simulator predicts what happens next. A differentiable one can answer which parameter should change to stop the failure.
Coding Agents Are Becoming General-Purpose Computer Workers
Repositories were a friendly environment: text in, terminal actions, checkable results. That was a starting point, not a boundary.
Open Standards Outlive Model Generations
A year after the MCP bet, the argument can be checked against what happened rather than what was hoped.
Colour Management Is a Pipeline Contract
Blender 5.0 reads as better display options. Giving a file an explicit working colour space is an architectural change.
The Best Model May Be a Router, Not a Model
GPT-5 moves model selection inside the system. The interesting unit stops being which model and becomes which compute policy.
World Models Are Not Game Engines Yet
Genie 3 generates a navigable 720p world at 24 fps. Production work needs state you can inspect when something goes wrong.
Memory Is Becoming a System Capability
A follow-up to the long-context argument. Storing, selecting and expiring facts is turning into a named part of the product.
Coding Agents Change the Unit of Software Work
AI coding tools have been judged where code appears on screen. The boundary moves when the agent owns a task instead of a snippet.
Agents Need Protocols Between Each Other, Not Just Tools
Tool calling solves the inside of the loop. It says nothing about one agent reaching another built by a different team on another platform.
Agents Need a Runtime, Not a Prompt Loop
An agent is no longer well described as a prompt inside a while loop. The useful abstraction owns execution around the model.
Reasoning Is Not the Only Path to Better Models
Longer thinking improves maths and code. A model that solves a logic puzzle and misreads ordinary intent is not the better production model.
Rendering Is Becoming a Reconstruction Stack
Sparse samples, motion data and lower-resolution frames become a larger final result. Debugging becomes layered when reconstruction sits in the middle.
Agent Reliability Is an Evaluation Problem, Not a Prompting Problem
When an agent misses a step, the usual fix is a stricter prompt. The failure is more often in how completion is detected.
MCP Might Matter More Than Another Model Release
A model can reason well and still be useless inside a company if it cannot reach the files, repositories and tools where work lives.
Computer Use Is the Missing Layer Between Models and Software
Most integrations assume useful software exposes the right API. Much of real software never did.
Inference-Time Compute Is a New Scaling Axis
o1 improves when it is allowed to spend longer on a problem. A benchmark score without a compute budget is an incomplete number.
The Final Pixel Won't Come From the Renderer
Geometry, camera and scene structure stay reliable ground truth. More of final appearance is moving into learned systems.
Open Models Are Becoming Research Infrastructure
Llama 3.1 gets discussed as a benchmark result. The licence terms change which experiments are possible at all.
The Model Is Becoming a Runtime
Function calling, code execution and structured output turn inference into a loop. The model stops being a text generator and starts being a control layer.
Multimodality Changes the Architecture, Not Just the Interface
GPT-4o is easy to read as a faster interface. Training one model end to end across text, vision and audio is an architectural change.
Benchmark Scores Are Not Model Capability
Claude 3 posts 86.8% on MMLU and 50.4% on GPQA Diamond. The chart is useful and it is not the same thing as capability.
Long Context Is Not Memory
Gemini 1.5 makes a million tokens usable. A larger working set is not a system that decides what should survive the session.