Robust and Sparse CCA: An Algorithm for Dimension Reduction via Sparsity Inducing Penalties.

Abstract

CCA (Canonical Correlation Analysis) is widely applied to measure the association between multivariate data sets, but the classical method is neither robust in the presence of atypical observations, nor does it lead to sparse canonical vectors, and thus it is not suitable for high-dimensional data with more variables than observations. While there are several approaches to achieve robustness or sparsity, only the alternating regression method proposed by Wilms & Croux [2015] combines both objectives. Higher-order canonical correlations, however, cannot be derived directly using this algorithm. We propose to reformulate the CCA objectives as an optimization problem with constraints, which allows for a direct statement of regularization conditions and a flexible choice of a covariance estimator. Let x and y denote a p-and q-dimensional random variable, respectively, and Σxx, Σyy and Σxy the corresponding covariance matrices. The first canonical correlation coefficient ρ1 and the first pair of canonical vectors (a1, b1) are given as a solution of the optimization problem.