Bias correction for linear discriminant analysis

Abstract

Abstract Linear discriminant analysis (LDA) is perhaps one of the most fundamental statistical pattern recognition techniques. In this work, we explicitly present, for the first time, an asymptotically exact estimator of the LDA optimal intercept in terms of achieving the lowest overall risk in the classification of two multivariate Gaussian distributions with a common covariance matrix and arbitrary misclassification costs. The proposed estimator of the optimal bias term is developed based on the theory of random matrices of increasing dimension in which the observation dimension and the sample size tend to infinity while keeping their magnitudes comparable. The simple form of this estimator provides us with some analytical insights into the working mechanism of the bias correction in LDA. We then complement these analytical insights with numerical experiments. In particular, empirical results using real data show that insofar as the overall risk is concerned, the proposed bias-corrected form of LDA can outperform the conventional LDA classifier in a wide range of misclassification costs. At the same time, the superiority of the proposed form over LDA tends to be more evident as dimensionality or the ratio between class-specific costs increase.