Colour Segmentation-based Computation of Dense Optical Flow with Application to Video Object Segmentation

Abstract

We propose a method for the computation of dense optical flow fields between two images of a video sequence. In a preprocessing step, the algorithm segments the reference frame into regions of homogenous colour. The flow vectors inside such regions are supposed to vary smoothly, while motion boundaries are assumed to coincide with boundaries of those regions. Given an initial affine motion model for each segment, the algorithm extracts a set of affine layers, which represent the dominant image motion, in a clustering procedure. Each segment is then assigned to one of those layers in order to optimize a global cost function. The cost function aims at generating smooth flow fields and models occlusions in both images. Layer extraction and assignment are then iteratively applied until convergence. Since the algorithm assigns segments that undergo the same affine motion to the same layer, the proposed method can equivalently be regarded as motion segmentation algorithm. Strong experimental results are achieved, especially in regions of poor texture and close to motion boundaries, where conventional methods often show poor performance.