Introducing depth information to 2D segmentation output of neural network

Abstract

This thesis evaluates the effects depth information has on estimation outputs of neural networks by including the depth data as post processing step. The algorithm implemented looks for planes in the depth point cloud via RANSAC corresponding to the areas determined as floor or wall in the estimation results of the neural network. The RANSAC results are then used to correct the segmentation by the means of nearest neighbour search. The results are analysed on different threshold level, which define the smallest objects to be considered during analysis. As neural network the AdapNet++ [1], a 50 layerresidual NN, is used. Whilst the NN is trained on the Scannet dataset, the implementation of depth data is performed on the NYUv2 dataset. The results show an average increase of segmentation accuracy of 1.54% over the different thresholds, peaking at 2.73% for objects bigger than 10000 pixels. The overall prediction accuracy on pixel level decreases after the introduction of depthdata by -0.22%.