Random projections for classification with high-dimensional data

Abstract

We examine the binary classification problem in a challenging high-dimensional setting with correlated predictors, where the coefficients in a logistic regression model can vary from sparse to dense. In this work, we propose the use of a data-driven random projection matrix to reduce the original feature space. The random projection combines variables considering their respective effect on the class response and such that the regression coefficient can still be recovered. In a simulation exercise, we show that the proposed random projection produces significantly better prediction results than conventional random projections, even outperforming benchmarks such as glmnet’s logistic regression with elastic net penalty.