Spatio-temporal filtering for real-time path tracing in virtual reality

Abstract

Ray tracing has raised its importance for real-time rendering since the presentation of NVIDIAs Turing architecture and Microsofts extension DirectX Raytracing (DXR). It allows for computing physically correct renderings dynamically for various global lighting effects such as shadows or reflection for example. Integration into major game engines such as Unreal Engine3 and Unity4 made these features quickly accessible to a wide range of game developers and graphics programmers. At the same time, consumer-level virtual reality (VR) systems gained more attention not just in the entertainment industry but also for medical, educational and training purposes. Perception of virtual scenes depends not only on geometric complexity but also heavily on physically plausible lighting. Hence, the combination of aforementioned real-time ray tracing with VR to produce visually more plausible lighting scenarios and increase immersion seems natural. Unfortunately, rendering for VR systems requires more computational power than ordinary graphics applications for desktop systems and real-time ray tracing is still limited in its extent.This thesis investigates if the light transport method - path tracing - is suitable to be used in a virtual reality setup in order to produce higher quality dynamic lighting. Therefore, a hybrid rendering pipeline is proposed combining path tracing with a low number of samples per pixel and spatio-temporal filtering of the noisy path tracer output. The pipeline comprises rasterization of first hits, tracing of a low amount of indirect rays per pixel, temporal accumulation of path tracing samples and, finally, de-noising of the highly noisy path traced image. In addition, this thesis introduces a novel masking system which provides a trade-off between the number of traced rays and the visual quality of dynamic objects. Furthermore, an improvement to the variance estimation of the chosen de-noising approach is proposed. A prototype shows that path tracing together with de-noising can achieve interactive frame rates suitable for VR applications. Resulting quality is comparable to reference renderings. The proposed masking system allows for further tuning of performance without introducing noticeable artifacts, making this approach viable for a large range of hardware setups.