Algorithms for 3D film content generation and post-processing

Abstract

In this talk we present algorithms for 3D reconstruction and novel view synthesis that form part of a 3D film processing chain which aims at generating high-quality 3D content for different types of 3D displays. The first part of the talk focuses on a trifocal camera set-up consisting of a high-end main camera and a compact assistant stereo rig which are aligned in an L-shape configuration. We discuss different strategies for stereo matching between the main and assistant cameras along with challenges posed by our specific camera arrangement. We further address the need for depth map post-processing and inpainting techniques in order to generate high-quality novel views. The results of our trifocal system with virtual view synthesis are evaluated by a user study. In the second part of the talk, we concentrate on 3D content generation from monoscopic film material. We describe a method for converting original 2D image sequences to 3D content based on comfortable user scribbles placed on key frames. The initially sparse depth values are propagated throughout the video volume to obtain temporally and perceptually coherent 2D-to-3D conversions. The results of this approach are illustrated on a variety of test material.