Writing
The Scene Graph Is Becoming an API for AI
A scene graph exists for artists and software. Agents are becoming another consumer, and they need structure rather than pixels.
Notes
For most 3D production, a scene graph exists for artists and software. It stores objects, transforms, materials, variants, animation and relationships in a form that DCC tools can compose and edit.
AI agents are beginning to become another consumer of that structure.
At SIGGRAPH this week, the Alliance for OpenUSD positioned OpenUSD as a shared 3D data layer for agentic workflows. Cesium is making geospatial data readable to agents. PTC is using multi-agent workflows to prepare CAD models for robotics simulation. NVIDIA is experimenting with agent-guided USD tooling, while creative applications are exposing scene operations through MCP.
I think this changes the role of the scene description layer.
If an agent is expected to inspect, reason about and modify a 3D world, pixels are not enough. It needs an API to what the world means.
A screenshot is the wrong abstraction for a structured scene
Computer-use models can already operate graphical interfaces from screenshots. That is useful when no better interface exists.
For a 3D application, though, the screen throws away information the application already knows. A viewport can show a mesh, but not reliably expose ownership, hierarchy, variants, inherited transforms, material source or production metadata.
OpenUSD represents those relationships directly.
A USD stage is a composed scene graph made from prims, attributes, relationships, layers, references, variants and time-sampled values. The Core Specification now defines those composition semantics as an open standard.
For an agent, that means structured state can be queried directly instead of inferred only from pixels.
Interoperability becomes machine understanding
USD was built to solve a production problem: large scenes need to move and compose across tools without flattening every department's work into one file.
That already made it valuable for film, VFX, design and simulation. Agentic workflows create another reason to want the same representation: an AI system needs stable names, hierarchy, types and relationships if it is expected to operate on a scene safely.
The Alliance for OpenUSD now describes the format as a shared, queryable representation for agents working across robotics, simulation and geospatial systems. Its July update points to early implementations where agents can navigate and modify environments represented through OpenUSD.
The old question was whether Blender, Maya, Houdini, Unreal or another tool could agree on the scene. The new question is whether a model can understand enough of that same contract to act on it.
Structured state gives agents something pixels cannot
Consider a simple production request:
"Find every hero product using the old material variant, update it to the approved version, but leave the archive collection untouched."
A vision model looking at the viewport can make guesses. A scene-aware agent can ask much better questions.
It can read which prims are hero products, which material binding is active, whether the material is inherited through a reference, which collection or layer represents the archive, whether the object is authored locally or arrives from another asset, and which edit target should receive the change.
These are scene-graph questions, not image-understanding questions.
OpenUSD gives the agent addressable prims, relationships, variants and layers that preserve where scene opinions come from. That does not make the model correct automatically, but it makes operations much more precise than telling an agent to click the third material dropdown it sees on screen.
A scene graph can become a capability boundary
This connects to a larger pattern in agent architecture.
The safest agent systems expose structured capabilities rather than unrestricted interfaces whenever possible. An agent should call validate_asset instead of rediscovering all validation logic. It should query project state rather than infer it from a screenshot.
A scene graph can provide a similar boundary for 3D.
The model does not need arbitrary access to every internal object in a DCC application. A tool layer can expose selected USD operations: inspect prims, read metadata, resolve references, switch variants, create overrides, validate schemas, or prepare a simulation-ready asset.
MCP can sit above that layer and expose those operations to the agent. OpenUSD can define the data the operations act on.
Those two layers solve different problems: MCP describes how the agent reaches the capability; OpenUSD describes the world that capability understands. That combination is more interesting to me than adding a chat panel to a 3D application.
NVIDIA's SIGGRAPH work is already moving in this direction, with MCP connections into creative tools and OpenUSD-based workflows for agent-assisted graphics and simulation.
Standardisation matters more once agents write back
Editing is where representation quality matters. OpenUSD's composition model lets multiple layers contribute to one final scene without destroying the original sources, which is useful for agent workflows.
I would not want an agent rewriting the original asset whenever it makes a suggestion.
A safer workflow is to let it author into a separate layer or override, then inspect the diff. If the change is wrong, remove the layer. If the change is approved, merge it through the normal pipeline.
The agent can act without becoming the unquestioned owner of the source data.
The Core Specification matters here because independent tools need to agree on how composition, value resolution and scene population behave. If agent-generated edits are expected to move across applications, those semantics cannot depend on one vendor's private interpretation.
OpenUSD is widening beyond static visual assets
The 2026 roadmap makes the trend broader.
OpenUSD 26.03 added native schema support for 3D Gaussian splats, allowing captured radiance representations to participate in the same composition system as traditional scene primitives. The Alliance has expanded work around characters, motion and interactivity.
The 26.08 release adds Profiles, multiple level-of-detail support and backplates, while continuing work on execution, splines and authoring infrastructure. Profiles are especially interesting because they allow applications to reason about which USD capabilities a scene or prim requires.
That looks useful for automation. A pipeline cannot assume every application supports every USD feature. A capability profile gives software, including agents, a machine-readable way to ask whether an asset fits a target environment before attempting a conversion or publish step.
What this means for Blender and production pipelines
For Blender workflows, I do not think this means every scene should suddenly become USD-first.
.blend remains a richer native representation for many Blender-specific workflows. The point is about boundaries.
When assets need to move between Blender, simulation, an engine, a renderer or an agent-controlled production service, a shared scene description becomes more valuable.
I would expose semantic information deliberately: stable object identities, material intent, asset metadata, variants, coordinate conventions and relationships that downstream systems can understand.
That makes the same asset easier for both software and agents to operate. Metadata becomes part of the language an AI system uses to understand what an asset is and what it is allowed to change.
The scene graph may become the machine interface to a world
World models can infer environments from pixels and computer-use agents can navigate screenshots, but production systems already possess something more reliable: explicit scene state.
If AI becomes deeply integrated into 3D production, I expect the most dependable agents to combine perception with structured access. Vision can help interpret what looks wrong. The scene graph can tell the system what actually exists and where an edit belongs.
That separation mirrors the rendering direction I have been writing about for two years.
Pixels are good for appearance.
Structured 3D state is better for truth.
My prediction
I think scene description will become part of AI infrastructure.
OpenUSD may become one of the strongest candidates because it already solves hierarchy, composition, references, variants, time-sampled data and cross-application interchange at production scale. The new push toward agentic workflows gives those properties another use.
I do not expect models to manipulate raw USD blindly. Domain tools, permissions and validation should sit between the model and the scene. Nor do I expect one format to cover every application-specific concept.
If an AI agent is expected to work inside a digital world, that world needs a machine-readable contract.
For years, the scene graph was an API between 3D tools.
It is starting to become an API for AI too.
Sources
- Alliance for OpenUSD, Alliance for OpenUSD Drives Global 3D Data Interoperability and Agentic AI Workflows with New Core Specification Milestones and Members, 21 July 2026.
- Alliance for OpenUSD, Announcing OpenUSD v26.08: Key Features and Improvements, 21 July 2026.
- Alliance for OpenUSD, USD Core Specification, 2026.
- NVIDIA, At SIGGRAPH, NVIDIA Advances Graphics and Simulation With Agentic and Physical AI, 20 July 2026.
- Pixar / OpenUSD, Universal Scene Description Core API Documentation, 2026.
More
Other write-ups
Approval Is Not Publication
Thirty seven items in the queue, one approved, nothing published. The last arrow in the diagram is the only one that pays.
Neural Rendering Is Crossing From Reconstruction Into Synthesis
Reconstruction filled in what sparse sampling missed. DLSS 5 generates appearance the renderer never computed.
The Renderer Is Becoming a Training Data Engine
The renderer used to sit at the end of the pipeline. Physical AI gives the same scene a second job: teaching a model.
Local AI Is Becoming a Compute Fabric
Local AI meant one model on one machine. Routing inference across the devices already on a network changes the unit.
Agent Infrastructure Is Becoming a Product Category
Every team used to build the loop, the store, the sandbox. That layer is being sold rather than written.
Choosing a local model with a stopwatch, not a benchmark
Three models, five hard tasks, code actually executed. The official build was 2.4 times faster and more accurate than a community repack of the same model.
We measured real time lighting against baked light, and baked won
A 3D product simulation that had to look like an offline render. The real time version ran at 60 frames per second and looked like clay. Here is the measurement and the architecture that replaced it.
What actually broke in an agency run by agents
Three failures from a delivery stack that runs on AI. None of them were the model's fault, and all three reported success while producing nothing.
A Task Without A Check Command Is Not Automated
If a task has no command that can fail, the pipeline advances on the appearance of work.
A Gate The Model Writes Is A Gate The Model Loosens
Three quality gates returned green while the work behind them was wrong, each for a different reason.
The Scoring Model Was Wrong And It Put The Worst Lead First
A weighted sum let one axis substitute for the other, so a company with money and no problem ranked in the top twenty.
Building Software Got Easy. Getting Value Out Of It Did Not
Aristo took weeks to build. Everything after the build is still in progress, and that gap is the whole story.
Capability Is Becoming an Operational Risk Surface
Safety questions used to be about the text. Once a model can act, the capability itself becomes something to operate.
Animation Is Moving From Clips to Motion Priors
Authored keyframes and blended clips are giving way to asking which constraints define acceptable motion.
Materials Are Becoming Learned Programs
A material is texture maps, parameters and shader code. It is starting to become a small learned program that answers a rendering question.
Procedural Systems Are Expanding Beyond Geometry
Geometry Nodes started with a narrow name. Blender 5.2 puts physics, sound and object data through the same graph.
The Unit of AI Work Is Becoming the Task, Not the Turn
Chat taught us to think one turn at a time. Long-running agents make the task the thing that is scheduled, resumed and reviewed.
Game Engines Are Becoming Operating Systems for Worlds
Engines have been judged on what they render and simulate. The Unreal 6 roadmap points at operating a world rather than drawing one.
The Model Is Becoming a Replaceable Backend
Choosing a provider used to mean choosing an architecture. A stable interface makes replacement possible and evaluation makes it safe.
The Harness Is Part of the Capability
The same model behaves differently depending on context policy, tool design and execution feedback. That surrounding software is not neutral.
Physics Engines Are Becoming Trainable Components
A simulator predicts what happens next. A differentiable one can answer which parameter should change to stop the failure.
The Agent Needs an Environment, Not Just Tools
A search function and a database query were enough for short loops. Longer work needs a place to stand.
Coding Agents Are Becoming General-Purpose Computer Workers
Repositories were a friendly environment: text in, terminal actions, checkable results. That was a starting point, not a boundary.
Open Standards Outlive Model Generations
A year after the MCP bet, the argument can be checked against what happened rather than what was hoped.
Colour Management Is a Pipeline Contract
Blender 5.0 reads as better display options. Giving a file an explicit working colour space is an architectural change.
The Best Model May Be a Router, Not a Model
GPT-5 moves model selection inside the system. The interesting unit stops being which model and becomes which compute policy.
World Models Are Not Game Engines Yet
Genie 3 generates a navigable 720p world at 24 fps. Production work needs state you can inspect when something goes wrong.
Memory Is Becoming a System Capability
A follow-up to the long-context argument. Storing, selecting and expiring facts is turning into a named part of the product.
Coding Agents Change the Unit of Software Work
AI coding tools have been judged where code appears on screen. The boundary moves when the agent owns a task instead of a snippet.
Agents Need Protocols Between Each Other, Not Just Tools
Tool calling solves the inside of the loop. It says nothing about one agent reaching another built by a different team on another platform.
Agents Need a Runtime, Not a Prompt Loop
An agent is no longer well described as a prompt inside a while loop. The useful abstraction owns execution around the model.
Reasoning Is Not the Only Path to Better Models
Longer thinking improves maths and code. A model that solves a logic puzzle and misreads ordinary intent is not the better production model.
Rendering Is Becoming a Reconstruction Stack
Sparse samples, motion data and lower-resolution frames become a larger final result. Debugging becomes layered when reconstruction sits in the middle.
Agent Reliability Is an Evaluation Problem, Not a Prompting Problem
When an agent misses a step, the usual fix is a stricter prompt. The failure is more often in how completion is detected.
MCP Might Matter More Than Another Model Release
A model can reason well and still be useless inside a company if it cannot reach the files, repositories and tools where work lives.
Computer Use Is the Missing Layer Between Models and Software
Most integrations assume useful software exposes the right API. Much of real software never did.
Inference-Time Compute Is a New Scaling Axis
o1 improves when it is allowed to spend longer on a problem. A benchmark score without a compute budget is an incomplete number.
The Final Pixel Won't Come From the Renderer
Geometry, camera and scene structure stay reliable ground truth. More of final appearance is moving into learned systems.
Open Models Are Becoming Research Infrastructure
Llama 3.1 gets discussed as a benchmark result. The licence terms change which experiments are possible at all.
The Model Is Becoming a Runtime
Function calling, code execution and structured output turn inference into a loop. The model stops being a text generator and starts being a control layer.
Multimodality Changes the Architecture, Not Just the Interface
GPT-4o is easy to read as a faster interface. Training one model end to end across text, vision and audio is an architectural change.
Benchmark Scores Are Not Model Capability
Claude 3 posts 86.8% on MMLU and 50.4% on GPQA Diamond. The chart is useful and it is not the same thing as capability.
Long Context Is Not Memory
Gemini 1.5 makes a million tokens usable. A larger working set is not a system that decides what should survive the session.