NVIDIA DLSS 3.5 announced: Revolutionary ray reconstruction technology for all GeForce RTX cards

As part of this year’s Gamescom 2023, NVIDIA has unveiled a number of upcoming updates aimed at the GeForce gaming community. A highlight among these announcements is the introduction of DLSS 3.5 Ray Reconstruction, a feature that will be available for all GeForce RTX series GPUs starting with the RTX 20 series in the upcoming fall season. DLSS 3.5 introduces a new element called “Ray Reconstruction Technology” designed to improve the visual quality of raytracing-based components in gaming scenarios.

While traditional rasterization meticulously calculates every pixel for every frame, real-time ray tracing encounters performance limitations that prevent it from taking this approach. Instead, only a limited number of rays are cast over a coarse grid during rendering, resulting in intermittent “black” gaps in the ray outputs. To counteract this, a noise reduction mechanism (“denoiser”) is used that runs various algorithms to effectively fill these gaps.

With the introduction of DLSS 3.5, NVIDIA introduces an optimized noise reduction mechanism that works seamlessly with DLSS 2 upscaling technology. This collaboration results in improved image quality results that are also more accurate. This particular feature leverages the Tensor Cores (as opposed to the RT Cores) and is therefore compatible with all GeForce RTX graphics cards ranging from the Turing architecture on up.

The image below illustrates the traditional approach to rendering real-time (RT) effects. It is important to note that in this scenario, DLSS 2 upscaling is enabled. First, the image is composited at a lower resolution and then upscaled to its original dimensions.

In the first step, the engine generates geometric shapes and material properties without applying shading effects. This information serves as the basis for building the BVH (Bounding Volume Hierarchy) acceleration structure, which is used in the context of ray tracing to help determine how rays interact with the geometric layout of the virtual environment. Subsequently, a series of rays are projected and their paths are carefully traced to calculate intersections, potential reflections, and possibly multiple interactions.

These results are then processed by the denoiser, a component that converts individual pixel data into a coherent image and simulates the appearance of raytracing-based phenomena such as reflections, shadows, illumination, or ambient occlusion. When upscaling is enabled, the denoiser produces output at a reduced rendering resolution, not the final native output resolution. The denoiser is not aware of the final resolution. The complexity is further increased by an additional challenge in that the upscaling process has no knowledge of rays; it only receives the pixel-based output from the denoiser without retaining the original ray tracing data at this point.

The central problem regarding denoisers is that they rely on preceding image sequences to collect sufficient pixel data for the final image. As a result, the output of raytracing-based (RT) content comes from the combination of multiple previous image sequences. The visual representation provided above illustrates situations where such challenges become apparent. To explain this in more detail, one can imagine the scenario of a moving car mirror that gathers information from multiple image sequences, creating undesirable visual artifacts such as ghosting. There are also concerns about subtle lighting effects and reflections that may appear distorted or diffuse due to this method.

NVIDIA’s innovation in DLSS 3.5 focuses on integrating the denoising and upscaling phases into a unified process. This unified process uses a broader data set to improve the quality of the output image. The results of lower resolution processing are merged with the results of rasterization, ray tracing, and motion vectors. These combined components are then integrated directly into a higher resolution image, such as a 4K image. Similar to DLSS 2, the DLSS 3.5 algorithm also draws on insights from previous image sequences through temporal feedback. After upscaling is complete, an additional cycle is performed for the DLSS 3 frame generation function, if applicable.

Presented here are findings shared by NVIDIA that show how DLSS 3.5 Ray Reconstruction, increase RT accuracy compared to traditional noise reduction techniques.

Ray Reconstruction has negligible impact on performance. In frame rate comparisons performed on an RTX 40-series GPU from NVIDIA, DLSS 3.5 RR offers slightly higher frame rates than DLSS 3 FG. NVIDIA has made it clear that DLSS 3.5 is not a performance enhancement feature, but the focus is on image quality. Depending on the scene, performance will be virtually identical, slightly better, or slightly worse. Theoretically, it is possible that game developers will reduce the number of beams when DLSS 3.5 is enabled. This could reduce the impact on RT performance and improve frame rates – still with improved image quality. However, there is no explicit support for this. This is exclusively a feature for game developers and does not fall within the scope of NVIDIA’s DLSS 3.5 implementation.

DLSS 3.5 will not only be available in games, but also in NVIDIA’s professional D5 renderer. There it will provide the ability to create real-time previews with impressive detail.

In the fall, DLSS 3.5 will be available on all GeForce RTX GPUs through a driver update. DLSS 3.5 now brings three distinct subcategories: Super Resolution (SR), the core frame magnification technology; Frame Generation (FG), introduced with DLSS 3, which uses AI to double frame rates by generating alternate frames; and now the novel Ray Reconstruction (RR) feature. DLSS 3.5 RR will work on all RTX GPUs, as all generations include Tensor Cores. For older RTX-20 “Turing” and RTX-30 “Ampere” cards, DLSS 3.5 will work the same on the latest RTX-40 “Ada” cards, but FG will not be available. Games that support Ray Reconstruction will have an additional “Enable Ray Reconstruction” option, similar to the existing “Enable Frame Generation” option. We have received confirmation from NVIDIA that DLAA is supported in combination with Ray Reconstruction. According to this, it is not necessary to use the upscaler all the time.

Although the designations may cause confusion, it is encouraging to see that NVIDIA is continuously working on the development of their technology. So far, there hasn’t been much news regarding AMD’s FSR 3; announcements to that effect may be made at Gamescom. From a technical perspective, Ray Reconstruction could rather be categorized as “DLSS 2.5”, as it has nothing to do with DLSS 3 frame generation and is strongly linked to DLSS 2 upscaling. It seems NVIDIA is now releasing all similar technologies under the established “DLSS” brand name and continuing to group them by their respective features. As an example, “DLSS 3 Frame Generation” is only supported on the GeForce 40 – this announcement doesn’t change that. However, the new “DLSS 3 Ray Reconstruction” works on the GeForce 20 and newer, similar to how “DLSS 2 Upscaling” also works on the GeForce 20.

Source: NVIDIA