Writing
Physics Engines Are Becoming Trainable Components
A simulator predicts what happens next. A differentiable one can answer which parameter should change to stop the failure.
Writing
A simulator predicts what happens next. A differentiable one can answer which parameter should change to stop the failure.
Notes
For most 3D and game-development work, a physics engine has one basic job: given the current state and some forces, predict what happens next.
That is forward simulation. It is how rigid bodies fall, cloth deforms, vehicles move and characters interact with the world.
Newton 1.0, released at GTC this week, points toward a broader role. The open-source engine from NVIDIA, Google DeepMind and Disney Research is built on Warp and OpenUSD, supports several solver families, and is designed for robot learning as much as conventional simulation. One of its defining features is differentiable physics: the simulation can provide gradients that describe how a result changes when an input, control signal or physical parameter changes.
I think that distinction will matter beyond robotics.
A physics engine is starting to become something an optimiser can reason through, not only something a user can run forward.
A traditional solver takes state at time t and advances it.
Give it positions, velocities, masses, constraints, contacts and forces. It integrates the system and produces the next state. Repeat that process and you have a simulation.
Newton 1.0 is built around that reality rather than one universal solver. NVIDIA describes a modular architecture containing several physics backends and numerical methods. MuJoCo Warp targets fast rigid-body simulation. Kamino handles mechanisms with closed loops. Vertex Block Descent can work with deformable materials such as cloth and cables, while material-point methods can handle granular systems such as sand.
Differentiability adds another dimension.
Suppose a robot arm drops an object.
A normal simulator can reproduce the failure if I give it the same initial conditions. It can tell me where the object moved and when contact occurred.
A differentiable simulator can help answer a different question: which parameters should change to reduce the failure?
If the simulation is differentiable, an objective at the end of the trajectory can propagate information backward through the physical computation. The system can compute how sensitive the outcome was to control inputs, friction, mass, geometry parameters or other variables represented in the simulation.
That turns physics into part of an optimisation graph.
Warp is built for this kind of workflow. NVIDIA's March update shows GPU simulation kernels written in Python, compiled for high-performance execution and differentiated automatically so they can participate in reverse-mode optimisation. The same simulation code can therefore generate a physical trajectory and provide gradients for a learning or design algorithm.
This changes the relationship between simulation and AI.
In many reinforcement-learning systems, simulation acts as an environment. An agent performs an action, the simulator returns the next state and a reward, and the learning algorithm gradually discovers a useful policy through repeated experience.
Differentiable physics gives us another path. When the problem allows useful gradients, parts of the learning signal can pass through the physical system itself.
Newton was designed around that connection. NVIDIA describes differentiability as a way to improve policy optimisation, design optimisation and system identification. The engine integrates with Isaac Lab and MuJoCo-based workflows, so physics is no longer an isolated module sitting beside machine learning. It can become one of the computations inside the learning system.
The simulator stops being only the place where we test a policy. It can participate in producing the policy.
There is another use that feels immediately practical: recovering physical parameters from observed behaviour.
A simulation is only as useful as its assumptions. If friction, stiffness, damping or mass distribution are wrong, the virtual result can diverge from the real system even when the solver itself is accurate.
System identification works backward from observations. Instead of asking what a known system will do, we ask which physical parameters would produce the behaviour we observed.
With differentiable simulation, that inverse problem can become an optimisation problem. Start with parameter estimates, simulate the system, compare the result with measurements, propagate the error backward, then update the parameters. This is one of the established application areas for differentiable simulators.
Imagine recording a real piece of fabric, soft packaging or a mechanical object, then fitting simulation parameters until the virtual behaviour matches the footage. The artist is no longer tuning stiffness and damping entirely by eye. The simulation can help infer them from reference data.
Choose a solver. Set mass. Adjust collision margins. Tune cloth stiffness. Run the result. Look at what failed. Change values. Run it again.
Differentiable simulation suggests a different workflow for some tasks.
The artist defines the target behaviour and the parameters the system is allowed to modify. The optimiser searches for a configuration that produces the desired result.
It could remove a large amount of parameter hunting where the desired target is measurable.
Physics traditionally sat outside the learning graph. The model could produce an action and the simulator could evaluate it, but the boundary between them often behaved like a black box.
Frameworks such as Warp reduce that separation. Simulation kernels can exchange data directly with PyTorch and JAX workflows while still expressing the control flow and sparse operations common in physical solvers.
That is a more interesting hybrid than asking a neural model to rediscover all of physics from data.
A simulator can provide precise gradients of the wrong model.
If the contact model is inaccurate, if the material representation is too simple, or if the real system contains behaviour the simulator does not represent, gradient descent can optimise toward a solution that works beautifully in simulation and fails outside it.
Contact itself is difficult because physical events can be discontinuous. Small changes can create or remove collisions. Friction introduces its own numerical problems. Differentiable simulators make engineering trade-offs around versatility, numerical behaviour and gradient quality.
It is a stronger optimisation interface to a model of reality. The quality of that model still matters.
For game development, I expect the near-term impact to be stronger in tooling than runtime gameplay.
A studio could optimise character controllers, vehicle parameters, procedural rigs or physically based interactions offline, then ship the resulting parameters into a conventional runtime. The player does not need gradient computation every frame for the training process to improve the final game system.
For Blender and product visualisation, inverse problems may be more useful than autonomous control. Matching physical parameters to captured reference, optimising a deformable setup toward a target shape, or finding design values that satisfy mechanical constraints could turn simulation into a production assistant rather than only a preview system.
The common idea is that simulation output stops being the end of the calculation.
It becomes feedback for the calculation that produced it.
I think physics engines will gradually become trainable components in larger computational systems.
More solvers will expose gradients, parameter estimation and optimisation hooks as normal interfaces. Simulation will sit inside loops that tune controllers, infer material properties, optimise designs and train models.
That could blur another boundary in production software. The renderer is becoming part of neural reconstruction. The scene graph is becoming an interface for agents. The physics engine may become part of the learning graph.
I would not expect every game or Blender simulation to become differentiable. Many workloads do not need it, and some physical problems are poorly suited to gradient-based optimisation.
The architectural direction still matters.
A traditional physics engine tells us what the world will do when we change an input.
A differentiable one can begin telling the system how the input should change.
More