Photorealistic Material Editing Through Direct Image Manipulation

Abstract

Creating photorealistic materials for light transport algorithms requires carefully fine-tuning a set of material properties toachieve a desired artistic effect. This is typically a lengthy process that involves a trained artist with specialized knowledge.In this work, we present a technique that aims to empower novice and intermediate-level users to synthesize high-qualityphotorealistic materials by only requiring basic image processing knowledge. In the proposed workflow, the user starts withan input image and applies a few intuitive transforms (e.g., colorization, image inpainting) within a 2D image editor of theirchoice, and in the next step, our technique produces a photorealistic result that approximates this target image. Our methodcombines the advantages of a neural network-augmented optimizer and an encoder neural network to produce high-qualityoutput results within 30 seconds. We also demonstrate that it is resilient against poorly-edited target images and propose asimple extension to predict image sequences with a strict time budget of 1-2 seconds per image