Sampling-Distribution-Based Evaluation for Monte Carlo Rendering

Abstract

In this paper, we investigate the application of per-pixel difference metrics for evaluating Monte Carlo (MC) rendering techniques. In particular, we propose to take the sampling distribution of the mean (SDM) into account for this purpose. We establish the theoretical background and analyze other per-pixel difference metrics, such as the absolute deviation (AD) and the mean squared error (MSE) in relation to the SDM. Based on insights from this analysis, we propose a new, alternative, and particularly easy-to-use approach, which builds on the SDM and facilitates meaningful comparisons of MC rendering techniques on a per-pixel basis. In order to demonstrate the usefulness of our approach, we compare it to commonly used metrics based on a variety of images computed with different rendering techniques. Our evaluation reveals limitations of commonly used metrics, in particular regarding the detection of differences between renderings that might be difficult to detect otherwise—this circ umstance is particularly apparent in comparison to the MSE calculated for each pixel. Our results indicate the potential of SDM-based approaches to reveal differences between MC renderers that might be caused by conceptual or implementation-related issues. Thus, we understand our approach as a way to facilitate the development and evaluation of rendering techniques.