Overscan Detection in Digitized Analog Films by Precise Sprocket Hole Segmentation

Abstract

Automatic video analysis is explored in order to understand and interpret real-world scenes automatically. For digitized historical analog films, this process is influenced by the video quality, video composition or scan artifacts called overscanning. The main aim of this paper is to find the Sprocket Holes (SH) in digitized analog film frames in order to drop unwanted overscan areas and extract the correct scaled final frame content which includes the most significant frame information. The outcome of this investigation proposes a precise overscan detection pipeline which combines the advantages of supervised segmentation networks such as DeepLabV3 with an unsupervised Gaussian Mixture Model for fine-grained segmentation based on histogram features. Furthermore, this exploration demonstrates the strength of using low-level backbone features in combination with low-cost CNN architectures like SqueezeNet in terms of inference runtime and segmentation performance. Moreover, a pipeline for creating photo-realistic frame samples to build a self-generated dataset is introduced and used in the training and validation phase. This dataset consists of 15000 image-mask pairs including synthetically created and deformed SHs with respect to the exact film reel layout geometry. Finally, the approach is evaluated by using real-world historical film frames including original SHs and deformations such as scratches, cracks or wet splices. The proposed approach reaches a Mean Intersection over Union (mIoU) score of 0.9509 (@threshold: 0.5) as well as a Dice Coefficient of 0.974 (@threshold: 0.5) and outperforms state-of-the-art solutions. Finally, we provide full access to our source code as well as the self-generated dataset in order to promote further research on digitized analog film analysis and fine-grained object segmentation.