Intel’s Frame Generation ExtraSS Technology for XeSS: A Potential Revolution in FPS Boosting

Holy smoke! Intel’s ExtraSS and XeSS Framegeneration could be game-changers! They might have some wild new tricks up their sleeves that could seriously boost frame rates. Would it be a real contender for FSR 3, or even DLSS 3 Frame Gen? only time will tell.

While Intel’s XeSS upscaling technology offers a glimpse into the possibilities of AI-powered graphics enhancement, its broader adoption within the gaming community has yet to fully flourish. Enter ExtraSS, a novel approach from Intel that seeks to push the boundaries of real-time rendering further.

Understanding XeSS and Advent of Extra SS

Before diving into ExtraSS, let’s revisit what XeSS offers. XeSS uses a trained AI model combined with motion vectors and frame history to intelligently upscale frames to full HD, 1440p, or 4K. This technology starts rendering every frame at a lower size than your target resolution, allowing for quicker rendering. Then, XeSS steps in to upscale these frames.

There are two versions of XeSS that you need to be aware of. The first one is open-sourced and works on any graphics card that supports DP4A instructions. DP4A is an instruction set supported by a range of GPUs that help accelerate artificial intelligence calculations.

The second version of XeSS takes advantage of Intel’s XMX units (AI-boosting hardware cores that are the equivalent to NVIDIA’s Tensor cores), included in the Intel Arc A770 and A750, to achieve a better-quality result. This version is exclusive to Intel cards, as it uses proprietary hardware that cards from other manufacturers lack.

While XeSS has been providing great image quality and wide support, it currently lacks a frame-generation feature. This is where ExtraSS comes into play. ExtraSS, a framework for joint spatial super-sampling and frame extrapolation, could potentially boost the game’s FPS.

XeSS uses a trained AI model combined with motion vectors and frame history to intelligently upscale frames. This technology starts rendering every frame at a lower size than your target resolution, allowing for quicker rendering. Then, XeSS steps in to upscale these frames.

The upscaling process involves synthesizing images that are very close to the quality of native ultra-high-res rendering by reconstructing subpixel details from neighboring pixels, as well as motion-compensated previous frames. This reconstruction is performed by a neural network trained to deliver high performance and exceptional quality.

We introduced ExtraSS, a novel framework that combines spatial super sampling and frame extrapolation to enhance real-time rendering performance. By integrating these techniques, our approach achieves a balance between performance and quality, generating temporally stable and high-quality, high-resolution results.

Leveraging lightweight modules on warping and the ExtraSSNet for refinement, we exploit spatial-temporal information, improve rendering sharpness, handle moving shadings accurately, and generate temporally stable results. Computational costs are significantly reduced compared to traditional rendering methods, enabling higher frame rates and alias-free high resolution results.

Evaluation using Unreal Engine demonstrates the benefits of our framework over conventional individual spatial or temporal super sampling methods, delivering improved rendering speed and visual quality. With its ability to generate temporary, stable, high-quality results, our framework creates new possibilities for real-time rendering applications, advancing the boundaries of performance and photo-realistic rendering in various domains.

via SIGGRAPH ASIA 2023

ExtraSS, Frame extrapolation In Depth

Frame extrapolation is a technique used to increase the apparent frame rate of a video. It generates new frames based on the existing ones, filling in the gaps between them. This can result in smoother gameplay and reduced input lag.

Frame extrapolation is another way to increase the framerate by only using the information from prior frames. Li et al.[2022] proposed an optical flow-based method to predict flow based on previous flows and then warp the current frame to the next frame. ExtraNet [Guo et al .2021] uses occlusion motion vectors with neural networks to handle dis-occluded areas and shading changes with G-buffers information. Their methods fail when the scene becomes complex and generate artifacts in the disoccluded areas. Furthermore, it requires higher resolution inputs since they only generate new frames. We are the first ones to propose a joint framework to solve both spatial supersampling and frame extrapolation together while staying efficient and high quality.

Interpolation vs. Extrapolation

Frame interpolation and extrapolation are two key methods of Temporal Super Sampling. Usually frame interpolation generates better results but also brings latency when generating the frames. Note that there are some existing methods such as NVIDIA Reflex [NVIDIA2020 ]decreasing the latency byusing a better scheduler for the inputs, but they cannot avoid the latency introduced from the frame interpolation and is orthogonal to the interpolation and extrapolation methods.

The interpolation methods still have larger latency even with those techniques. Frame extrapolation has less latency but has difficulty handling the disoccluded areas because of lacking information from the input frames. Our method proposes a new warping method with a lightweight flow model to extrapolate frames with better qualities to the previous frame generation methods and less latency comparing to interpolation based methods.

Intel Research Paper (ExtraSS: A Framework for Joint Spatial Super Sampling and Frame Extrapolation)

The introduction of ExtraSS could significantly enhance the gaming experience. By combining the benefits of XeSS’s intelligent upscaling and ExtraSS’s frame extrapolation, gamers could enjoy high-quality visuals without compromising on performance. This could potentially revolutionize the gaming industry, setting a new standard for game FPS.

Stay tuned for more updates on Intel’s Frame Generation Technology for XeSS and the upcoming ExtraSS.