Yoshitatsu Matsuda

Analysis of computer science related curriculum on LDA and Isomap

A good curriculum is crucial for a successful university education. When developing a curriculum, topics, such as natural science, informatics, and so on are set first, course syllabi are written accordingly. However, the topics actually by the courses are not guaranteed to be identical to the initially set topics. To find out if the actual topics are covered by the developed course syllabi, we developed a method of systematically analyzing syllabi that uses latent Dirichlet allocation (LDA) and Isomap. We applied this method to the syllabi of MIT and those of the Open University, and verified that the method is effective.

Linear multilayer ICA generating hierarchical edge detectors

In this letter, a new ICA algorithm, linear multilayer ICA (LMICA), is proposed. There are two phases in each layer of LMICA. One is the mapping phase, where a two-dimensional mapping is formed by moving more highly correlated (nonindependent) signals closer with the stochastic multidimensional scaling network. Another is the local-ICA phase, where each neighbor (namely, highly correlated) pair of signals in the mapping is separated by MaxKurt algorithm. Because in LMICA only a small number of highly correlated pairs have to be separated, it can extract edge detectors efficiently from natural scenes. We conducted numerical experiments and verified that LMICA generates hierarchical edge detectors from large-size natural scenes.

An adaptive threshold in joint approximate diagonalization by assuming exponentially distributed errors

Joint approximate diagonalization (JAD) is one of well-known methods for solving blind source separation. JAD diagonalizes many cumulant matrices of given observed signals as accurately as possible, where the optimization for each pair of signals is repeated until the convergence. In each pair optimization, JAD should decide whether the pair is actually optimized by a convergence decision condition, where a fixed threshold has been employed in many cases. Though a sufficiently small threshold is desirable for the accuracy of results, the speed of convergence is quite slow if the threshold is too small. In this paper, we propose a new decision condition with an adaptive threshold for JAD under a probabilistic framework. First, it is assumed that the errors in JAD (non-diagonal elements in cumulant matrices) are given by the exponential distribution. Next, it is shown that the maximum likelihood estimation of the probabilistic model is equivalent to JAD. Then, an adaptive threshold is theoretically derived by utilizing the model selection theory. Numerical experiments verify the efficiency of the proposed method for blind source separation of artificial sources and natural images. It is also shown that the proposed method is especially effective when the number of samples is limited.

An efficient MDS-based topographic mapping algorithm

Here, an multidimensional scaling-based (MDS-based) topographic mapping algorithm is proposed, named the stochastic MDS network. Because this network utilizes not local but global information over all the units, it can find more optimal results than previous models. In addition, by using a stochastic gradient algorithm, the mapping formation in this network is carried out as efficiently as in SOM-like models based on only the local information. Some simple numerical experiments verified the validity and efficiency of this network. It was also applied to the formation of large-scale topographic mappings, and could form various interesting mappings.

GLOBAL MAPPING ANALYSIS: STOCHASTIC APPROXIMATION FOR MULTIDIMENSIONAL SCALING

In this paper, we propose global mapping analysis (GMA) as a new method to solve multidimensional scaling (MDS). By GMA, MDS is done by an online learning rule based on stochastic approximation. GMA need not directly calculate the disparity matrix for carrying out MDS, as Ojas PCA network do not calculate the correlation matrix. So, GMA is expected to be useful for multivariate data analysis on a large scale. Actually, it was verified by numerical experiments based on artificial data that GMA can work well even if the number of the attribute N is quite large (N=10,000.)

Development of a curriculum analysis tool

A good curriculum is crucial for a successful university education. When developing a curriculum, topics, such as economics, natural science, informatics, etc. are set first, and course syllabi are written accordingly. However, the topics actually covered by the course syllabi are not guaranteed to be identical to the initially set topics. To find out if the actual topics covered by the developed course syllabi, we developed a method of systematically analyzing course syllabi that uses latent Dirichlet allocation (LDA) and Isomap. In this paper, we propose the web-based curriculum analysis tool with this method, and demonstrate an example of the way the tool is used for analyzing computer science curricula.

Joint Approximate Diagonalization Utilizing AIC-Based Decision in the Jacobi Method

Joint approximate diagonalization is one of well-known methods for solving independent component analysis and blind source separation. It calculates an orthonormal separating matrix which diagonalizes many cumulant matrices of given observed signals as accurately as possible. It has been known that such diagonalization can be carried out efficiently by the Jacobi method, where the optimization for each pair of signals is repeated until the convergence of the whole separating matrix. Generally, the Jacobi method decides whether the optimization is actually applied to a given pair by a convergence decision condition. Then, the whole convergence is achieved when no pair is actually optimized any more. Though this decision condition is crucial for accelerating the speed of the whole optimization, many previous works have employed simple conditions based on an arbitrarily selected threshold. In this paper, we propose a novel decision condition which is based on Akaike information criterion (AIC). It is derived by assuming each cumulant matrix to be a sample generated independently. In each pair optimization, the condition compares the reduction rate of the objective function with a constant depending on the number of cumulant matrices. It involves no thresholds (and no parameters) to be set manually. Numerical experiments verify that the proposed decision condition can accelerate the optimization speed for artificial data.

Linear mltilayer ICA using adaptive PCA

Linear multilayer independent component analysis (LMICA) is an approximate algorithm for ICA. In LMICA, approximate independent components are efficiently estimated by optimizing only highly dependent pairs of signals when all the sources are super-Gaussian. In this paper, the nonlinear functions in LMICA are generalized, and a new method using adaptive PCA is proposed for the selection of pairs of highly dependent signals. In this method, at first, all the signals are sorted along the first principal axis of their higher-order correlation matrix. Then, the sorted signals are divided into two groups so that relatively highly correlated signals are collected in each group. Lastly, each of them is sorted recursively. This process is repeated until each group consists of only one or two signals. Because a well-known adaptive PCA algorithm named PAST is utilized for calculating the first principal axis, this method is quite simple and efficient. Some numerical experiments verify the effectiveness of LMICA with this improvement.

The infomin principle for ICA and topographic mappings

It has been well known that edge filters in the visual system can be generated by the InfoMax principle. In this paper, the “InfoMin” principle is proposed, which asserts that the information through some neighboring signals on a two-dimensional mapping must be minimized. It is shown that the standard Comon’s ICA can be derived from the combination of the InfoMax principle for the whole signals and the InfoMin one for each signal under a linear model with sufficiently large noise. It is also shown that the InfoMin principle for the signals within neighboring areas can generate a topographic mapping in the same way as in topographic ICA.

A fixed-point algorithm of topographic ICA

Topographic ICA is a well-known ICA-based technique, which generates a topographic mapping consisting of edge detectors from natural scenes. Topographic ICA uses a complicated criterion derived from a two-layer generative model and minimizes it by a gradient descent algorithm. In this paper, we propose a new simple criterion for topographic ICA and construct a fixed-point algorithm minimizing it. Our algorithm can be regarded as an expansion of the well-known fast ICA algorithm to topographic ICA, and it does not need any tuning of the stepsize. Numerical experiments show that our fixed-point algorithm can generate topographic mappings similar to those in topographic ICA.