Advanced bayesian estimation in hierarchical Gaussian models: Dirichlet process mixtures and clustering gain

Abstract

This Master’s thesis explores the use of a Dirichlet process (DP) prior to enhance Bayesian estimation of the parameters of multiple objects. Specifically, it focuses on a hierarchical Gaussian model where each object is linked to one parameter of interest, one noisy measurement, and one hyperparameter, and the hyperparameters are shared among objects within the same cluster. The model permits the derivation of closed-form performance bounds, enabling a quantification of performance improvements relative to the theoretically achievable performance. Our primary objective is to estimate the parameter of interest for each object based on its associated noisy measurement while leveraging the cluster structure of the hyperparameters. Because a closed-form calculation of the posterior distribution is not possible, we employ a Markov chain Monte Carlo sampling method to approximate the minimum mean square error (MMSE) estimator. This methodology yields an estimator that exploits the inherent cluster structure and, as we show through simulations, consistently achieves a mean squared error (MSE) that is lower than the MSE of the MMSE estimator for a scenario without a cluster structure. Additionally, we derive a closed-form MMSE estimator assuming known object-cluster associations and demonstrate its performance through simulations. Our approach of combining estimation and clustering demonstrates superior performance compared to the widely used method of first clustering and then performing estimation within each cluster; however, this performance advantage comes at the cost of a higher computational complexity.