Radiant future? NVIDIA DLSS 3.5 and the Next Level of Raytracing

Released today, NVIDIA DLSS 3.5 is the latest iteration of NVIDIA’s Deep Learning Super Sampling (DLSS) technology and brings a number of enhancements and new features to improve the ray tracing experience in games and applications. In doing so, NVIDIA has set ambitious goals to also increase adoption of the new technology. True to the motto: “The eye eats with you”. Which brings us to the goal of today’s article, because I will (almost) exclusively deal with the optics today.

Speaking of acceptance and mass suitability: Today’s article was deliberately created with a GeForce RTX 4070, because a GeForce RTX 4090 can do much more, but it is certainly nothing that represents the broad mass of readers. That is why the screenshots and videos were mostly created in WQHD and UWQHD, respectively, even though Ultra HD naturally looks finer. But we deliberately focus on what the average user can take away from it in practice. And that’s where DLSS in Performance Mode comes into play, which benefits more from the new Ray Reconstruction than Quality Mode.

Ray Reconstruction (RR)

But I don’t want to leave you in the rain without some theory or background information, you have to go through it now. So once again the most important question: “What is Ray Reconstruction? It is, in a nutshell, a new AI model that creates higher quality images for intensive RT games and applications. It replaces hand-tuned denoisers with an AI network trained by NVIDIA supercomputers that creates higher-quality pixels between sampled rays, making games more realistic and beautiful. This method uses high-frequency sampled rays to add sufficient information to a frame computed in 1080p that is later needed for output in Ultra HD, for example. This is achieved by adding vector information from the game itself

Traditional denoisers (we even know this from Blender & Co.) can cause problems like ghosting, removing dynamic effects and reducing the quality of other aspects. They also tend to blur too much detail unnecessarily or, conversely, don’t always blur enough when they do need to. This is a shortcoming that leads to non-uniform lighting effects. “Ray Reconstruction” effectively addresses these problems by making smarter decisions about the use of temporal and spatial data. It can identify specific ray tracing effects and integrate other game information such as motion vectors and surface colors to produce a much more accurate final image in ray tracing games. This results in significantly improved clarity, accuracy and sharpness of ray tracing effects.

The AI model has been trained with 5 times more data than DLSS 3, allowing it to make smarter decisions about the use of temporal and spatial data and provide high-frequency information for superior upscaling quality. In today’s example with Cyberpunk 2077, “Ray Reconstruction” improves the quality and clarity of reflections, making global illumination more accurate and lighting more dynamically responsive. The feature detects specific patterns associated with reflections and keeps the image stable while collecting accurate pixels and blending adjacent pixels to create high-quality reflections. For developers, the technology simplifies the development process by elegantly eliminating the need for manually tuned denoisers. For gamers, this means improved image quality with more realistic and immersive experiences. So it’s kind of a win-win for game developers and gamers. And of course NVIDIA, let’s not forget.

 

Super Resolution and DLAA

Super Resolution is a key technology in NVIDIA’s DLSS 3.5 suite, based on (what else?) artificial intelligence (AI) capabilities. This technology leverages the Tensor Cores of GeForce RTX GPUs to create images that are of equivalent or even higher quality than native resolution. This not only speeds up frame rates and improves image quality, but also increases responsiveness in games and applications. Developers have also had the ability to integrate Super Resolution into their projects quite easily and quickly for some time now by downloading the appropriate DLSS SDKs and plugins. Another advantage of this technology is its wide availability, as it is at least supported on all RTX GPUs.

DLAA, short for Deep Learning Anti-Aliasing, is a technology that uses the AI techniques also present in DLSS to improve image quality. Unlike DLSS, which reduces the resolution and then uses an AI model to recover the image, DLAA works directly on the native resolution to improve edge smoothing. This means that it does not increase the frame rate, but may actually cost a bit of power to achieve higher image quality. So you can use DLSS or DLAA, but never both at the same time, which is a completely different approach.

 

DLAA is particularly useful in scenarios where the frame rate is already very high and users are willing to sacrifice a few frames per second (fps) to achieve improved image quality. It should be noted, however, that games must explicitly support this technology to take advantage of DLAA. This technology is part of NVIDIA’s latest DLSS 3.5 update and is designed to improve the visual quality of games by significantly improving edge smoothing without reducing native resolution. This results in images that are sharper, more detailed, and negatively affected with fewer visual artifacts such as “ghosting.”

Frame Generation and NVIDIA Reflex

These technologies work together to boost performance and make gameplay more responsive. Frame Generation leverages the power of NVIDIA’s Ada Lovelace GPU tensor cores and Optical Flow Accelerator. But we’ve already covered these technologies in detail, so you can read that all again without me having to blow up today’s article immensely.

Availability and integration

A small side note, which NVIDIA always packs in our lunch box: After all, over 330 published games and applications now support NVIDIA DLSS, including over 40 that now support DLSS 3.5. The NVIDIA Streamline SDK makes it easier for programmers to integrate DLSS 3.5 and includes Super Resolution as well as DLAA (which is available for all RTX GPUs), as well as Frame Generation for the Ada cards. and Ray Reconstruction, which is also available for all RTX GPUs.

Developers who have already integrated DLSS 2 with NVIDIA’s Streamline SDK are in the clear, as it’s then just a tiny step to integrate DLSS 3 and its various components. The Streamline SDK is an open cross-IHV solution that simplifies the integration of the latest NVIDIA and other independent hardware vendor super-resolution technologies into applications and games. It supports a variety of technologies, including DLAA, NVIDIA Image Scaling (NIS), NVIDIA Real-Time Denoisers (NRD), DLSS Ray Reconstruction, NVIDIA Reflex, and DLSS Frame Generation.

And because so many always complain about the supposedly so proprietary Cyberpunk 2077 as the new Crysis: Unreal Engine 5.2 now also supports DLSS 3. The Unreal Engine is a popular tool for creating real-time 3D content, i.e. interactive experiences and immersive virtual worlds. How the whole thing will look qualitatively in UE 5.x, of course, remains to be seen, but that there is even an easy-to-use plugin for the UE, lets hope for the best.