Kei Hirose

Sparse estimation via nonconcave penalized likelihood in factor analysis model

We consider the problem of sparse estimation in a factor analysis model. A traditional estimation procedure in use is the following two-step approach: the model is estimated by maximum likelihood method and then a rotation technique is utilized to find sparse factor loadings. However, the maximum likelihood estimates cannot be obtained when the number of variables is much larger than the number of observations. Furthermore, even if the maximum likelihood estimates are available, the rotation technique does not often produce a sufficiently sparse solution. In order to handle these problems, this paper introduces a penalized likelihood procedure that imposes a nonconvex penalty on the factor loadings. We show that the penalized likelihood procedure can be viewed as a generalization of the traditional two-step approach, and the proposed methodology can produce sparser solutions than the rotation technique. A new algorithm via the EM algorithm along with coordinate descent is introduced to compute the entire solution path, which permits the application to a wide variety of convex and nonconvex penalties. Monte Carlo simulations are conducted to investigate the performance of our modeling strategy. A real data example is also given to illustrate our procedure.

Bayesian Information Criterion and Selection of the Number of Factors in Factor Analysis Models

In maximum likelihood exploratory factor analysis, the estimates of unique variances can often turn out to be zero or negative, which makes no sense from a statistical point of view. In order to overcome this diculty, we employ a Bayesian approach by specifying a prior distribution for the variances of unique factors. The factor analysis model is estimated by EM algorithm, for which we provide the expectation and maximization steps within a general framework of EM algorithms. Crucial issues in Bayesian factor analysis model are the choice of adjusted parameters including the number of factors and also the hyper-parameters for the prior distribution. The choice of these parameters can be viewed as a model selection and evaluation problem. We derive a model selection criterion for evaluating a Bayesian factor analysis model. Monte Carlo simulations are conducted to investigate the eectiveness of the proposed procedure. A real data example is also given to illustrate our procedure. We observe that our modeling procedure prevents the occurrence of improper solutions and also chooses the appropriate number of factors objectively.

Estimation of an oblique structure via penalized likelihood factor analysis

The problem of sparse estimation via a lasso-type penalized likelihood procedure in a factor analysis model is considered. Typically, model estimation assumes that the common factors are orthogonal (i.e., uncorrelated). However, if the common factors are correlated, the lasso-type penalization method based on the orthogonal model frequently estimates an erroneous model. To overcome this problem, factor correlations are incorporated into the model. Together with parameters in the orthogonal model, these correlations are estimated by a maximum penalized likelihood procedure. Entire solutions are computed by the EM algorithm with a coordinate descent, enabling the application of a wide variety of convex and nonconvex penalties. The proposed method is applicable even when the number of variables exceeds that of observations. The effectiveness of the proposed strategy is evaluated by Monte Carlo simulations, and its utility is demonstrated through real data analysis.

Tuning parameter selection in sparse regression modeling

In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows Cp type criteria may be used as a tuning parameter selection tool in lasso type regularization methods, for which the concept of degrees of freedom plays a key role. In the present paper, we propose an efficient algorithm that computes the degrees of freedom by extending the generalized path seeking algorithm. Our procedure allows us to construct model selection criteria for evaluating models estimated by regularization with a wide variety of convex and nonconvex penalties. The proposed methodology is investigated through the analysis of real data and Monte Carlo simulations. Numerical results show that Cp criterion based on our algorithm performs well in various situations.

A comparison of parallel multipliers with neuron MOS and CMOS technologies

We intend to obtain a fast and high-density logic circuit combining neuron MOS transistors (neuMOS), that was developed in Tohoku university, into a binary logic circuit. In this paper, we focus on basic arithmetic functional circuits, a full-adder and a multiplier, and make a comparison of the area and delay of the neuMOS circuits with conventional CMOS logic circuits. The results of physical design and SPICE simulation show that the area of a neuMOS multiplier with full-adders decreases to about 65% of the area of CMOS, and the delay of a neuMOS multiplier with (7,3) parallel counters decreases to about 70% of the delay of CMOS.

Robust sparse Gaussian graphical modeling

Gaussian graphical modeling has been widely used to explore various network structures, such as gene regulatory networks and social networks. We often use a penalized maximum likelihood approach with the

NNRMLR: A Combined Method of Nearest Neighbor Regression and Multiple Linear Regression

L_1

Readouts for echo-state networks built using locally regularized orthogonal forward regression

penalty for learning a high-dimensional graphical model. However, the penalized maximum likelihood procedure is sensitive to outliers. To overcome this problem, we introduce a robust estimation procedure based on the

Full information maximum likelihood estimation in factor analysis with a large number of missing values

gamma

Variable selection via the weighted group lasso for factor analysis models

-divergence. The parameter estimation procedure is constructed using the Majorize-Minimization algorithm, which guarantees that the objective function monotonically decreases at each iteration. Extensive simulation studies showed that our procedure performed much better than the existing methods, in particular, when the contamination rate was large. Two real data analyses were carried out to illustrate the usefulness of our proposed procedure.