Depth and Thermal Images in Face Detection - A Detailed Comparison Between Image Modalities

Abstract

Face detection is a well-known issue in image processing, and numerous studies are present in this field. A prominent part of the work is devoted to RGB images, leaving depth and thermal data with less interest. However, in some conditions like low-light areas where face detection is needed, non-RGB sensors might perform better. Also, mounting an additional RGB camera could be challenging or not possible, considering privacy concerns. In this work, current deep learning methodologies are employed to train depth and thermal detection models. The training is done using combined publicly available data that is processed by us for this purpose in order to create necessary annotations for a learning process. The resulting models are validated on a new trimodal dataset collected for this experiments purpose. It contains images captured with RGB, depth, and thermal sensors. Various scenes with single and multiple faces appearances can be found. The results show that non-RGB solutions can be applied in practice with highly robust accuracy and their efficiency is close to RGB detectors. However, their performance depends on the environment and that circumstances are described later in this article.