A Fast Unified System for 3D Object Detection and Tracking

Abstract

We present FUS3D, a fast and lightweight system for real-time 3D object detection and tracking on edge devices. Our approach seamlessly integrates stages for 3D object detection and multi-object-tracking into a single, end-to- end trainable model. FUS3D is specially tuned for indoor 3D human behavior analysis, with target applications in Ambient Assisted Living (AAL) or surveillance. The system is optimized for inference on the edge, thus enabling sensor- near processing of potentially sensitive data. In addition, our system relies exclusively on the less privacy-intrusive 3D depth imaging modality, thus further highlighting the potential of our method for application in sensitive areas. FUS3D achieves best results when utilized in a joint detec- tion and tracking configuration. Nevertheless, the proposed detection stage can function as a fast standalone object de- tection model if required. We have evaluated FUS3D ex- tensively on the MIPT dataset and demonstrated its supe- rior performance over comparable existing state-of-the-art methods in terms of 3D object detection, multi-object track- ing, and, most importantly, runtime.