Variational inference for Dirichlet process mixtures and application to Gaussian estimation

Abstract

Bayesian nonparametric models have emerged as a flexible tool for learning patterns and structures in complex data. A well-known Bayesian nonparametric model is the Dirichlet process mixture (DPM) model. The DPM model extends the Bayesian mixture model with a finite number of mixture components to a Bayesian mixture model with a countably infinite number of mixture components while permitting the use of efficient Bayesian inference methods. Thereby, it becomes possible to cluster data and estimate unknown parameters without specifying the number of clusters and unknown parameters a priori. Practical Bayesian inference methods for DPM models include Markov chain Monte Carlo (MCMC) and variational inference (VI) methods. In this thesis, we focus on the VI methodology and provide a detailed derivation of the coordinate ascent variational inference (CAVI) algorithm for DPM models. Subsequently, we apply the CAVI algorithm to a Gaussian estimation problem that involves noisy observations of object features modeled by a DPM with Gaussian mixture components. We present simulation results that compare the improvement of the estimation accuracy of the object features due to clustering achieved by our CAVI algorithm and by a previously proposed MCMC algorithm. Our simulation results demonstrate to which extent we have to sacrifice estimation accuracy when opting for the less accurate but more efficient CAVI algorithm as opposed to the MCMC algorithm.