Attention-driven object detection and segmentation for robotics

Abstract

Vision is an essential part of any robotic system and plays an important role in such typical robotic tasks for domestic environments as searching and grasping objects in cluttered scenes. To be efficient, vision systems are required to provide fast object detection and segmentation mechanisms. In the past, attention mechanisms have been proposed to cope with the complexity of the real world by detecting and prioritizing the processing of objects of interest, and therefore guide the search and segmentation of objects. The goal of this thesis is to create an attention-based visual system, consisting of attention-based object detection and attention-driven object segmentation for a robot. Many models of visual attention have been proposed and proven to be very useful in robotic applications. We address the problem of obtaining meaningful saliency measures based on such characteristics as the object height and surface orientations that appear to be qualitatively better than traditional saliency maps. Moreover, recently it has been shown in the literature that not only single visual features, based on color, orientation or curvature attract attention, but complete objects do. Symmetry is a feature of many man-made and also natural objects and has thus been identified as a candidate for attentional operators. However, not many techniques exist to date that exploit symmetry-based saliency. In this thesis, a novel symmetry-based saliency operator that works on 3D data and does not assume any object model is presented. We show that the proposed saliency maps are better suited for the task of object detection. Object detection was implemented by means of extracting fixation points from saliency maps. The evaluation in terms of the quality of fixation points showed that the proposed algorithms outperform current state-of the-art saliency operators. The quality of attention points was defined in terms of their location within the object and the number of attended objects. Segmentation of highly cluttered indoor scenes is a challenging task and traditional segmentation methods are often overwhelmed by the complexity of the scene and require a significant amount of processing time. To tackle this problem we propose to use attention-driven and incremental segmentation, where attention mechanisms are used to prioritize parts of the scene to be handled first. In this work, we combined a saliency operator based on 3D symmetry with three segmentation methods. The first one is based on clustering locally planar surface patches. The second method segments attended objects using an edge map based on color, depth and curvature within a probabilistic framework. We also proposed a third method, an incremental attention-driven mechanism, that outputs object hypotheses composed of parametric surface models. We evaluated our approaches on two publicly available datasets of cluttered indoor scenes containing man-made objects. We showed that the proposed methods outperform existing state-of-the-art attention-driven segmentation algorithms in terms of segmentation quality and computational performance.