Manabu Kotani

Analysis of gene expression data by using self-organizing maps and k-means clustering

There is a growing need for a method to analyze massive gene expression data obtained from DNA microarrays. We introduce a method of combining a self-organizing map (SOM) and a k-means clustering for analyzing and categorizing the gene expression data. The SOM is an unsupervised neural network learning algorithm and forms a mapping the high-dimensional data to two-dimensional space. However, it is difficult to find clustering boundaries from results of the SOM. On the other hand, the k-means clustering can partition the data into the clusters under the assumption of the known number of clusters. In order to understand easily the results of SOM, we apply the k-means clustering to finding the clustering boundaries from results of SOM. We have applied the proposed method to the published data of yeast gene expression and show that the method is effective for categorizing the data.

Emergence of feature extraction function using genetic programming

A novel method of feature extraction to improve the performance of pattern recognition is proposed. It is assumed that the feature consists of a polynomial expression of the original patterns. The term of polynomial expressions are searched by genetic programming. In order to evaluate the effectiveness of the proposed method, we apply the k nearest neighbor classifier as the classification algorithm. Experiments were performed for an artificial task and an acoustic diagnosis for compressors as the real world task. From these results, we confirmed that the proposed method was effective for the feature extraction.

Feature Extraction Using Independent Components of Each Category

We describe an application of independent component analysis (ICA) to pattern recognition in order to evaluate the effectiveness of features extracted by ICA. We propose a recognition method suitable for independent components that consists of modules for each category. A module has two parts: feature extraction and classification. Features are independent components estimated by ICA and outputs of modules are candidates for categories. These candidates are combined and categories are decided with a majority rule. This recognition method is applied to two tasks: hand-written digits in the MNIST database and acoustic diagnosis for a compressor as real-world tasks. A FastICA algorithm is applied to extracting independent features in the proposed method. Through recognition experiments, we demonstrate that the ICA of each category extracts useful features for these tasks and the independent components are superior to the principal components in recognition accuracy.

Application of Kernel Principal Components Analysis to pattern recognitions

Kernel Principal Component Analysis (Kernel PCA) is one of the methods to perform PCA in high dimensional space. The purpose of this paper is to examine what components are obtained by Kernel PCA and evaluate effectiveness of the components as feature. Simulations results show that Kernel PCA can get superior performance to PCA.

A study of feature extraction using supervised independent component analysis

Recently, independent component analysis (ICA) has been applied to not only problems of blind signal separation, but also feature extraction of images and sounds. In this paper, we study the effectiveness of Umeyamas (1999) supervised ICA (SICA) for feature extraction of handwritten characters. Two types of control vectors (supervisor) are proposed for SICA: 1) average patterns (Type-I); and 2) eigen-patterns (Type-II). To demonstrate the usefulness of SICA, recognition performance is evaluated for handwritten digits that are included in the MNIST database. From the results of recognition experiments, we certify that SICAs with both types of control vectors work effective for feature extraction. Actually, the within-class variance between-class variance ratio of SICA features with Type-I control vectors becomes slightly larger as compared with a conventional ICA.

Application of independent component analysis to handwritten Japanese character recognition

We explore an approach to recognizing Japanese Hiragana characters utilizing independent components of input images (we call this method ICA-matching). These components are extracted by the fast ICA algorithm proposed by Hyvarinen and Oja (1997). We propose several formats of inputs, which are different in how a character image is transformed into time sequences. From recognition experiments, we show that ICA-matching outperforms conventional methods in some cases. However, in order to realize high performance, we focus on the following parameters: dimensions of feature vectors and the rate of noise added to the training data. The question of how these parameters are related to the performance of ICA-matching is discussed.

Detection of gas leakage sound using modular neural networks for unknown environments

It is important to detect flammable or poisonous gas leaked from the cracks in pipes of petroleum refining plants or chemical plants. We applied a novel strategy of construction of neural network to the acoustic diagnosis technique for the gas leakage. An example of the modular neural network to realize the strategy is able to adapt its structure according to the dynamic environment. Experiments were performed for an artificial gas leakage device under various experimental conditions over about 18 months in a petroleum refining plant. Experimental results showed that the proposed network could adapt the structure to changes in environments and its performance was superior to that of feed-forward networks with the re-training strategy. From these results, we confirmed the effectiveness of the modular neural network for practical use.

A structural learning algorithm for multi-layered neural networks

We propose a new structural learning algorithm for organizing the structure of the multi-layered neural networks. The proposed pruning algorithm consists of two already known algorithms, the structural learning algorithm with forgetting and the optimal brain damage algorithm using the second derivatives of the assessment. After the network is slimmed by the structural learning algorithm with forgetting, unimportant weights are pruned from the network using the second derivatives. The simulations are performed for the Boolean function and the acoustic diagnosis of compressors. The results show that the proposed algorithm is effective for eliminating the unimportant weights.

Feature extraction using supervised independent component analysis by maximizing class distance

Recently, Independent Component Analysis (ICA) has been applied to not only problems of blind signal separation, but also feature extraction of patterns. However, the effectiveness of features extracted by ICA (ICA features) has not been verified yet. As one of the reasons, it is considered that ICA features are obtained by increasing their independence rather than by increasing their class separability. Hence, we can expect that high-performance pattern features are obtained by introducing supervisor into conventional ICA algorithms such that the class separability of features is enhanced.. In this work, we propose SICA by maximizing Mahalanobis distance between classes. Moreover, we propose a new distance measure in which each ICA feature is weighted by the power of principal components consisting of the ICA feature. In the recognition experiments, we demonstrate that the better recognition accuracy for two data sets in UCI Machine Learning Repository is attained when using features extracted by the proposed SICA.

Acoustic diagnosis for compressor with hybrid neural network

Describes an acoustic diagnosis technique for a compressor using a hybrid neural network (HNN). The HNN is composed of two neural networks: an acoustic feature extraction network, and a fault discrimination network. The acoustic feature extraction network uses an auto-associative neural network (ANN) whose target patterns are the same as the input patterns. The five-layered neural network is composed of two three-layered neural networks to compress the input information and to restore the compressed information. The authors examine the architecture of the ANN for acoustic diagnosis, the proper form of the activation function in the output layer and the proper number of hidden layers. The fault discrimination network uses a multilayered neural network whose input patterns are the output values of the hidden layer in the ANN. The authors examine the possibility of discriminating between eight types of compressor faults with high accuracy by using an HNN.<<ETX>>