Sequential bayesian optical flow estimation using temporal coherence

Abstract

Optical flow (OF) estimation is an ambitious research field in computer vision that has met with growing interest and has produced an increasing number of publications during the last 35 years. Several algorithms for OF estimation have been developed, and their performance has been continuously improved. OF estimation has a large number of applications, such as the tracking of moving objects and the analysis of the dynamical behavior of objects in an image sequence. In this thesis, we review the classical deterministic model for OF estimation and subsequently reformulate it in a probabilistic framework. We then extend the probabilistic OF model to account for temporal coherence. With the concept of optimal Bayesian filtering as our basis, we derive the information form of the Kalman filter and the variational Bayesian filter, and we formulate these filters in the specific context of OF estimation. Finally, we study and compare the accuracy and computational complexity of the two filters for synthetic and real image sequences. In particular, we discuss the potential benefit of the temporal coherence assumption for different types of data.The original contributions of this thesis include the following:• We develop a probabilistic OF model that combines temporal coherence with a non-linear brightness constancy constraint.• We present a direct derivation of the information form of the Kalman filter and formulate the filter’s prediction and update steps in the context of temporally coherent OF estimation.• We derive a method for temporally coherent OF estimation based on variational Bayesian filtering.• We provide simulation results that assess and compare the accuracy and computational complexity of the presented OF estimation methods for several synthetic and real image sequences, and we investigate if and how much temporal coherence improves the results of OF estimation.