Interactive Rotated Object Detection for Novel Class Detection in Remotely Sensed Imagery

Abstract

In this paper, we propose IRTR-DETR, an Interactive and Real-Time Rotated DEtection TRansformer that extends IRT-DETR to predict rotated bounding boxes. IRTR-DETR maintains the Human-In-The-Loop (HIL) workflow of IRTDETR but introduces rotation-aware heads for improved detection of objects with arbitrary orientations. Similarly to IRTDETR, IRTR-DETR can be trained with a small labeled sample set in an interactive setting, but we show that it can also be pretrained on related but not identical data-such as a building damage dataset-before being applied to tasks like identifying buildings under construction. We demonstrate the efficacy of our approach on the publicly available Tiny-DOTA and xBD dataset, as well as two study-cases on proprietary datasets of greenhouses and houses under construction (“waffle homes”). Detecting greenhouses is highly relevant in the context of damage assessment, while “waffle homes” aid understanding typical floorplans and building codes in different areas, both thereby supporting population modeling, emergency response, and policy planning. Our method outperforms the state of the art in interactive rotated object detection on the Tiny-DOTA dataset by 5.7 percent, and improves upon the non interactive RTDETR by 7.85 to 19.39 percent (depending on the number of provided samples) while maintaining its real-time efficiency.