Class-centric visual interactive labeling using property measures

Abstract

Human annotation of image data is relevant for supervised machine learning, where labeled datasets are essential for training models. Traditionally, reducing the labeling effort was achieved through active learning, where the optimal next instance for labeling is selected by some heuristic to maximize utility. More recent work has focused on integrating user initiative in the labeling process through visual interactive labeling to steer the labeling process. This thesis proposes cVIL, a class-centric approach for visual interactive labeling that simplifies the human annotation process for large and complex image datasets. Previously, visual labeling approaches were typically instance-based, where the system visualizes individual instances for the user to label. cVIL utilizes diverse property measures to enable the labeling of difficult instances individually and in batches to label simpler cases rapidly. Since the property measures express the properties of an instance using a single scalar value, the visualizations are simple and scalable. cVIL combines the heuristic guidance approach of active learning with the user-centered approach of visual interactive labeling. In simulations, we could show that property measures can facilitate effective instance and batch labeling. In a user study, cVIL demonstrated superior accuracy and user satisfaction compared to the conventional instance-based visual interactive labeling approach that employs scatterplots. Participants also needed less time to complete the assigned tasks in cVIL compared to the baseline.