Automated Prioritization for Context-Aware Re-rendering in Editing

Abstract

Real-time Monte Carlo path tracing has become a feasible option for interactive 3D scene editing due to recent advancements in GPU ray tracing performance, as well as (AI-accelerated) denoising techniques. While it is thus gaining increasing support in popular modeling software, even minor edits such as adjusting materials or moving small objects typically require current solutions to discard previous samples and restart the image formation process from scratch. A recent solution introduced two adaptive, priority-based re-rendering techniques implementing incremental updates while focusing first on reconstructing regions of high importance and gradually addressing less critical areas. An extensive user study compared these prioritized renderings with conventional same-time re-rendering to evaluate their effectiveness for interactive scene editing. Results indicate a significant preference for incremental rendering techniques for editing small objects over traditional full-screen re-rendering with denoising, even with basic priority policies. Building upon these results, we revisit the underlying design choices and derive more sophisticated priority policies that respect global illumination effects (shadows and reflections) as well as employing attention-based techniques (based either on eye tracking to prioritize areas in the user’s gaze or, alternatively, using the cursor position).