A Variational Bayes Approach to Factor Analysis

Abstract

Factor analysis models are useful dimensionality-reduction techniques for the covariance of observed data. A Bayesian approach to inference for these models offers several benefits over the frequentist counterparts, including regularized estimates and inclusion of subjective prior information. However, implementation of Bayesian FA is routinely based on Markov Chain Monte Carlo (MCMC) techniques that are computationally expensive and often do not scale in high dimensions. To improve scalability, we propose a Variational Bayes algorithm that approximates the posterior distribution at a fraction of the MCMC computational cost. We compare the performance of variational Bayes and MCMC based algorithms in a simulation study and show that even in low dimensions variational Bayes is much faster than MCMC while still producing highly accurate point estimates.