NVIDIA Showcases Real-Time Neural Materials Models, Offering Up To 24x Shading Speedup

NVIDIA has unveiled an exciting breakthrough in real-time rendering with its “Neural Appearance Models.” Leveraging AI, these models promise a remarkable 12-24x speedup in shading performance compared to traditional methods. Let’s dive into the details.

The Need for Speed

At Siggraph, NVIDIA is turning heads with its cutting-edge approach. Last year, they introduced Neural Compression, which unlocked 16x texture detail. Now, they’re pushing the envelope further by enhancing texture rendering and shading performance.

Universal Runtime Mode

The Neural Appearance Models operate as a universal runtime mode for all materials. Whether sourced from real-world objects, artist captures, measurements, or text prompts via Generative AI, these models are scalable across various quality levels—from PC gaming to Virtual Reality and Film Rendering.

Capturing Every Detail

The neural models excel at capturing intricate details. Dust particles, water spots, lighting nuances, and even the interplay of light rays—all are meticulously rendered. Traditionally, shading graphs were used, but they proved costly and complex for real-time rendering.

The Neural Materials Approach

NVIDIA replaces the traditional materials rendering model with a more efficient neural network. This shift enables 12-24 times faster shading calculations. A side-by-side comparison demonstrates the Neural Materials model matching reference images with lightning speed.

Innovations Galore

The new model introduces several innovations:

1. Film-Quality Neural Materials: A complete and scalable system for film-grade materials.
2. Tractable Training: Efficient training for gigatexel-sized assets using an encoder.
3. Decoders with Priors: Normal mapping and sampling are optimized.
4. Efficient Execution: Neural networks run seamlessly in real-time shaders.

How Neural Models Work

During rendering, neural materials behave similarly to traditional models. At each hit point, textures are looked up, and two MLPs (Multi-Layer Perceptrons) evaluate the BRDF value and outgoing direction. Built-in graphics priors enhance inference quality, and the encoder handles massive resolutions.

High-Resolution Textures

All models rendered using Neural Materials offer up to 16K texture resolution. These detailed objects improve game performance while maintaining realism.

Hardware Support

NVIDIA leverages existing machine learning frameworks (PyTorch, TensorFlow), GLSL/HLSL tools, and hardware-accelerated matrix multiply-accumulate (MMA) engines. The open-source Slang shading language compiles neural material descriptions for Vulkan, Direct3D 12, and CUDA.

Tensor-Core Acceleration

Tensor-core architecture plays a crucial role. Although currently limited to compute APIs, NVIDIA exposes Tensor-core acceleration to shaders, optimizing code generation.

Performance Showcase

Using an NVIDIA GeForce RTX 4090 GPU with hardware-accelerated DXR (Ray Tracing) at 1920x1080, the Neural approach outperforms reference models. Full-frame rendering times demonstrate significant speedups.

A Game-Changer

NVIDIA’s Neural Materials models redefine real-time rendering. With their impressive speedup, developers and content creators can generate ultra-realistic textures efficiently. Stay tuned for their impact on upcoming games and apps!