Masashi Hyodo

Asymptotic expansion and estimation of EPMC for linear classification rules in high dimension

The problem of classifying a new observation vector into one of the two known groups distributed as multivariate normal with common covariance matrix is considered. In this paper, we handle the situation that the dimension, p, of the observation vectors is less than the total number, N, of observation vectors from the two groups, but both p and N tend to infinity with the same order. Since the inverse of the sample covariance matrix is close to an ill condition in this situation, it may be better to replace it with the inverse of the ridge-type estimator of the covariance matrix in the linear discriminant analysis (LDA). The resulting rule is called the ridge-type linear discriminant analysis (RLDA). The second-order expansion of the expected probability of misclassification (EPMC) for RLDA is derived, and the second-order unbiased estimator of EMPC is given. These results not only provide the corresponding conclusions for LDA, but also clarify the condition that RLDA improves on LDA in terms of EPMC. Finally, the performances of the second-order approximation and the unbiased estimator are investigated by simulation.

Multiple Comparisons Among Mean Vectors When the Dimension is Larger Than the Total Sample Size

We consider pairwise multiple comparisons and multiple comparisons with a control among mean vectors for high-dimensional data under the multivariate normality. For such cases, the statistics based on the Dempster trace criterion are given, and also their approximate upper percentiles are derived by using the Bonferroni’s inequality. Finally, the accuracy of their approximate values is evaluated by Monte Carlo simulation.

Modified Jarque–Bera Type Tests for Multivariate Normality in a High-Dimensional Framework

In this article, we introduce two types of new omnibus procedures for testing multivariate normality based on the sample measures of multivariate skewness and kurtosis. These characteristics, initially introduced by, for example, Mardia (1970) and Srivastava (1984), were then extended by Koizumi, Okamoto, and Seo (2009), who proposed the multivariate Jarque-Bera type test (MJB1) based on the Srivastava (1984) principal components measure scores of skewness and kurtosis. We suggest an improved MJB test (MJB2) that is based on the Wilson-Hilferty transform, and a modified MJB test (mMJB) that is based on the F-approximation to mMJB. Asymptotic properties of both tests are examined, assuming that both dimensionality and sample size go to infinity at the same rate. Our simulation study shows that the suggested mMJB test outperforms both MJB1 and MJB2 for a number of high-dimensional scenarios. The mMJB test is then used for testing multivariate normality of the real data digitalized character image.

The Asymptotic Approximation of EPMC for Linear Discriminant Rules Using a Moore-Penrose Inverse Matrix in High Dimension

We consider the discriminant rule in a high-dimensional setting, i.e., when the number of feature variables p is comparable to or larger than the number of observations N. The discriminant rule must be modified in order to cope with singular sample covariance matrix in high-dimension. One way to do so is by considering the Moor-Penrose inverse matrix. Recently, Srivastava (2006) proposed maximum likelihood ratio rule by using Moor-Penrose inverse matrix of sample covariance matrix. In this article, we consider the linear discriminant rule by using Moor-Penrose inverse matrix of sample covariance matrix (LDRMP). With the discriminant rule, the expected probability of misclassification (EPMC) is commonly used as measure of the classification accuracy. We investigate properties of EPMC for LDRMP in high-dimension as well as the one of the maximum likelihood rule given by Srivastava (2006). From our asymptotic results, we show that the classification accuracy of LDRMP depends on new distance. Additionally, our asymptotic result is verified by using the Monte Carlo simulation.

A Model Selection Criterion for Discriminant Analysis of Several Groups When the Dimension is Larger than the Total Sample Size

This article deals with a criterion for selection of variables for the multiple group discriminant analysis in high-dimensional data. The variable selection models considered for discriminant analysis in Fujikoshi (1985, 2002) are the ones based on additional information due to Rao (1948, 1970). Our criterion is based on Akaike information criterion (AIC) for this model. The AIC has been successfully used in the literature in model selection when the dimension p is smaller than the sample size N. However, the case when p > N has not been considered in the literature, because MLE can not be estimated corresponding to singularity of the within-group covariance matrix. A popular method used to address the singularity problem in high-dimensional classification is the regularized method, which replaces the within-group sample covariance matrix with a ridge-type covariance estimate to stabilize the estimate. In this article, we propose AIC-type criterion by replacing MLE of the within-group covariance matrix with ridge-type estimator. This idea follows Srivastava and Kubokawa (2008). Simulations revealed that our proposed criterion performs well.