Autonomous In-hand Object Modeling from a Mobile Manipulator

Abstract

Robots require knowledge of objects to manipulate and operate them in their environment. However, such object models are not always readily available and must first be created. Service robots are well-equipped to perform this autonomously, thanks to their set of sensors and arm. Once grasped, the object of interest can be captured under many angles and separated from the background, and the relative transformation between views can be measured through proprioceptive sensors. As no object knowledge is available, the approach needs to rely on knowledge of the robot’s own manipulator, and the environment stability during the manipulation. This work focuses on investigating different methods for segmenting objects moved by a mobile manipulator from captured RGB-D images, using knowledge of the arm and of the scene’s background. These segmented views are used to reconstruct the object, based on the arm forward kinematics and Iterative Closest Point (ICP) alignment of a 3D hand model with the scene. We examine the segmentation on different objects, and demonstrate that the proposed method provides accurate results even for transparent objects.