Interactive Object Detection for Tiny Objects in Large Remotely Sensed Images

Abstract

This paper highlights the potential of a Human-In-the-Loop (HIL) in interactive object detection methods. Although automation in computer vision is advancing rapidly, certain critical tasks, such as detecting UneXploded Ordnance (UXO), space/marine debris, or the generation of new datasets, require 100% recall and near-perfect precision. These tasks are often performed manually since automatic methods do not achieve the necessary accuracy. However, interactive object detection frameworks can potentially enhance annotation speed while maintaining the recall and accuracy of manual annotation. We propose IRTDETR, an interactive and real-time object detection method for very large imagery to address this. Using either point or bounding box annotations provided by a HIL, it globally relates the full image with the annotator inputs via a cross-attention-like mechanism, employs an attention loss to maximize the classification score based on similarity, and reuses portions of the network outputs during iterative refinements to conserve resources. We conduct experiments on five different datasets (Tiny-DOTA, CHAI, AITOD, SarDET, and COCO) to verify the efficacy of our approach. Our method surpasses existing interactive annotation approaches, achieving a higher mean Average Precision (mAP) with the same number of clicks. Additionally, we validate the annotation efficiency of our method in a user study, demonstrating it is 2.46× quicker and asks for only 72% of the task load (NASA-TLX) compared to fully manual annotation.