Neural Rendering Is Crossing From Reconstruction Into Synthesis

Two years ago, I wrote that the final pixel would not always come directly from the renderer. My argument was that geometry, camera state and scene structure would remain the reliable ground truth, while more of final appearance would move into learned systems.

Last year, DLSS 4 made that direction easier to see. Rendering was becoming a reconstruction stack: sparse physical samples, motion data and lower-resolution frames were being turned into a larger final result by neural stages.

DLSS 5 changes the boundary again.

NVIDIA now describes it as a real-time generative rendering stage. Unlike earlier DLSS systems, which reconstructed a higher-cost result that a conventional renderer could in principle have produced, DLSS 5 can add appearance detail learned from real-world visual data. NVIDIA points to effects such as skin subsurface scattering, light transmission through foliage and richer material response that may not be explicitly represented in the authored scene.

That is a different role for AI in graphics.

The neural stage is no longer only recovering missing samples. It is beginning to synthesise appearance.

Reconstruction had a reference target

Earlier DLSS systems were easier to describe. Super Resolution reconstructs a higher-resolution image, Ray Reconstruction approximates a more expensive lighting result, and Frame Generation creates intermediate frames.

In each case, the desired answer is still defined by the conventional rendering pipeline. Given enough samples and compute, the renderer could theoretically produce the reference result itself. Neural reconstruction exists because doing that directly is too expensive in real time.

DLSS 5 changes this assumption. NVIDIA's research team says the system complements conventional rendering with appearance priors learned from real-world visual data. The model can synthesise visual effects that the scene representation or real-time rendering budget does not fully describe.

In other words, there may no longer be a full-quality conventional frame waiting somewhere behind the approximation.

The neural system is contributing knowledge that is not contained entirely in the renderer.

This is why geometry still matters

The obvious failure mode of generative graphics is loss of control.

A normal image generator can create a convincing face while changing the person's identity. It can improve a material while moving a seam. It can make lighting more realistic while altering geometry. In a still image that may be acceptable. In a game or product visualisation pipeline, it is not.

DLSS 5 is built around this problem.

NVIDIA calls the approach 3D-Guided Neural Rendering. The model is conditioned on the current rendered frame, motion vectors, temporal state and artistic controls. During training, renderer-derived attributes are used to keep generation aligned with scene structure. NVIDIA says the inference process is deterministic and trained for temporal stability.

The game engine therefore remains the authority over what exists.

Geometry, proportions, camera projection, motion and authored composition define the constraints. The neural model operates inside them.

This is close to the architecture I expected when I argued that geometry would remain ground truth even as final appearance became learned. The renderer is not being removed; its output is becoming a constraint surface for another stage.

The renderer is turning into a control signal

This changes how I think about a rendered frame.

Traditionally, the frame is the product. Every earlier stage exists to produce those pixels.

In a generative rendering pipeline, the conventional frame can become an intermediate representation. It tells the neural stage where objects are, how they move, which surfaces are visible, how light is behaving and what the artist has already decided.

The final appearance is produced afterward.

NVIDIA's implementation takes colour and motion vectors at inference time, while the model is trained to recognise renderer information including surface albedo, lighting and normals. The system then adds learned lighting and material behaviour while preserving the structure of the source frame.

That means the renderer is increasingly useful because of the information it can guarantee, not because every output pixel must be final.

For technical artists, that is a meaningful shift. We may care less about whether one shader graph can reproduce every microscopic visual effect directly and more about whether the scene provides stable, semantically useful signals to the learned stage.

Art direction becomes the hard part

Generative systems introduce a new production problem: realism is not always the desired result.

A stylised game may intentionally avoid realistic skin. A product render may require a material to match an approved reference rather than whatever a model considers visually plausible. A character's likeness cannot drift because another appearance looks more natural.

NVIDIA is addressing this with developer controls rather than treating the model as a black box.

In NBA 2K27, developers can select different neural models for different scenes and use masks to control where the generative treatment is applied. NVIDIA's examples preserve underlying facial geometry while changing light response and material detail.

A production-ready neural renderer needs an art-direction interface. The question is not only what it can generate, but which properties artists can lock, vary and keep stable across thousands of frames. I expect this to become its own technical-art discipline.

What changes for asset production

If neural rendering continues in this direction, I do not think 3D artists stop building assets. I think the value of different asset properties changes.

Shape, silhouette, scale, deformation, material segmentation, motion and semantic structure remain valuable because they constrain the generated result. Some expensive micro-level appearance may move downstream into learned rendering.

That does not mean AI will texture everything. Production needs repeatability, exact materials and stable art direction. The likely trade is narrower: artists author what must be controlled, while learned systems handle more of the expensive perceptual tail.

This will affect Blender workflows too

DLSS 5 is a real-time game technology, but the architectural direction applies beyond games.

In Blender, I already separate structural truth from final appearance. Geometry determines the object. Cameras determine framing. Materials and lighting describe physical intent. The renderer turns that scene into samples.

A future neural stage could consume those same signals and produce a final image with more visual richness than the scene explicitly encodes.

For product visualisation, that is attractive and dangerous. It could reduce the work required to model microscopic appearance, but commercial use would need strong controls around shape, logos, colour and material identity.

A plausible pixel is not enough. It has to remain true to the product.

The distinction between simulation and appearance is getting sharper

What interests me most is that graphics may be separating into two problems.

The first is simulation: what exists, where it is, how it moves, what is visible and what physical constraints apply.

The second is appearance: given those constraints, what should the final image look like?

Traditional rendering couples these tightly. Learned rendering can separate them more aggressively.

That could let engines spend deterministic compute where truth matters and generative compute where perceptual realism is expensive.

I would expect different applications to draw that boundary differently. Competitive games need stable, predictable outputs. Film may accept more expensive learned stages. Product visualisation will prioritise fidelity to the source asset. Stylised games may deliberately restrict generative realism.

The technology does not remove art direction. It makes the boundary between authored truth and synthesised appearance something teams have to design explicitly.

My prediction

I think neural rendering is now crossing from reconstruction into synthesis.

That is a stronger claim than I would have made in 2024, because the production evidence is different now. DLSS 5 is not simply estimating more rays or more pixels. NVIDIA says it is the first DLSS system that generates the final displayed appearance rather than reconstructing a higher-cost reference output from the conventional renderer. It runs locally as a real-time rendering stage on RTX 50 Series hardware and debuted commercially in NBA 2K27.

I still do not expect conventional rendering to disappear.

In fact, the more generative the final stage becomes, the more valuable trustworthy 3D structure may become. Geometry, motion, materials and lighting buffers give the model the constraints that general image generators lack.

My earlier prediction was that the renderer would stop being the sole author of the final pixel.

The more precise version now is this:

The renderer is becoming the source of truth.

The neural system is becoming the source of appearance.

Sources

1. NVIDIA ADLR, DLSS 5: Generative Neural Rendering, 1 September 2026.
2. NVIDIA GeForce, DLSS 5 3D-Guided Neural Rendering Debuts in NBA 2K27, 1 September 2026.