Learning image similarities from forensic evidence

Abstract

Forensic evidence, such as toolmarks, footwear impressions, and handwritten documents treated in this thesis, is crucial to solving criminal cases. Even though such forensic evidence is meticulously collected and digitalized, a manual search for matching evidence from different cases in an archive of hundreds to thousands of images is time-consuming. Therefore, this thesis presents a methodology for comparing and retrieving forensic images using automatic analysis of image similarities. In contrast to other image analysis tasks, like object classification, for forensic images, fine- grained local characteristics are crucial since they can uniquely identify the ob- ject or person that has left behind a trace on the crime scene. For toolmarks and footwear impressions, such characteristics occur, for example, due to damages or wear. Since they have the potential to yield the highest evidential strength, such individual characteristics are the most powerful during an examination. The proposed methodology facilitates metric learning to learn a similarity measure that is specific for each forensic domain addressed. This approach allows efficient comparison of local characteristics in a learned embedding space. In order to utilize the available data more effectively and provide a mechanism to enforce domain-specific constraints, methods for modeling the global context by combining local characteristics are presented. The proposed methodology is evaluated using datasets from the three exem- plary forensic image modalities addressed, i.e., toolmarks, footwear impressions, and handwritings. Further, two new publicly available datasets are presented, explicitly designed to train and evaluate learning-based methods for retrieving forensic toolmark images and footwear impressions.