Automated analysis of the Vienna ”Naturhistorisches Museum” Herbarium collection

Abstract

Herbaria are collections of preserved plant specimens, accompanied by metadata such as species classification, collection location, and collector information. This thesis applies deep learning techniques to analyze herbarium specimen images by utilizing both visual and textual data, including handwritten labels, printed metadata, and specimen features.The primary objective is to develop a self-supervised deep learning model using contrastive learning, which clusters similar data points without requiring labeled data. This approach addresses the variability in herbarium collections and enables robust plant classification into families, genera, and species. Unlike supervised methods, this approach leverages convolutional neural networks to learn meaningful representations of specimens without the need for explicit class labels during training. To enhance the analysis, Handwritten Text Recognition (HTR) and Optical Character Recognition (OCR) methods are applied to extract textual information from specimen labels. The accuracy of these methods is evaluated by comparing predicted text against ground truth data. A visualization tool is developed within this thesis to allow researchers to explore clusters of related specimens based on model embeddings. This tool facilitates the analysis of individual specimens and provides access to associated metadata for deeper insights. Additionally, a segmentation dataset for dried plant parts is created to improve image analysis.This work demonstrates the effectiveness of self-supervised learning in herbarium specimen classification, offering a scalable, generalizable alternative to supervised methods while supporting advanced botanical research.