Nobuhiko Yamaguchi

Combining classification improvements by ensemble processing

The k-nearest neighbor (KNN) classification is a simple and effective classification approach. However, improving performance of the classifier is still attractive. Combining multiple classifiers is an effective technique for improving accuracy. There are many general combining algorithms, such as Bagging, Boosting, or Error Correcting Output Coding that significantly improve the classifier such as decision trees, rule learners, or neural networks. Unfortunately, these combining methods developed do not improve the nearest neighbor classifiers. In this paper, first, we present a new approach to combine multiple KNN classifiers based on different distance functions, in which we apply multiple distance functions to improve the performance of the k-nearest neighbor classifier. Second, we develop a combining method, in which the weights of the distance function are learnt by genetic algorithm. Finally, combining classifiers in error correcting output coding, are discussed. The proposed algorithms seek to increase generalization accuracy when compared to the basic k-nearest neighbor algorithm. Experiments have been conducted on some benchmark datasets from the UCI machine learning repository. The results show that the proposed algorithms improve the performance of the k-nearest neighbor classification.

Self-organizing hidden Markov models

The self-organizing mixture models (SOMMs) were proposed as an expectation-maximization (EM) algorithm that yields topology preserving maps of data based on probabilistic mixture models. Compared to self-organizing maps, the SOMM algorithm has a clear interpretation: it maximizes the sum of data log likelihood and a penalty term that enforces self-organization. The object of this paper is to extend the SOMM algorithm to deal with multivariate time series. The standard SOMM algorithm assumes that the data are independent and identically distributed samples. However, the i.i.d. assumption is clearly inappropriate for time series. In this paper we propose the extension of the SOMM algorithm for multivariate time series, which we call self-organizing hidden Markov models (SOHMMs), by assuming that the time series is generated by hidden Markov models (HMMs).

Variational Bayesian inference with Automatic Relevance Determination for Generative Topographic Mapping

Generative Topographic Mapping (GTM) is a nonlinear latent variable model introduced by Bishop et al. as a data visualization technique. In this paper, we focus on variational Bayesian inference for the GTM. The variational Bayesian GTM was first proposed by Olier et al. However, the GTM of Olier et al. uses a single regularization term and regularization parameter to avoid overfitting and therefore cannot locally control the degree of regularization. To overcome the problem, we propose the variational Bayesian inference with Automatic Relevance Determination (ARD) hierarchical prior for the GTM. The proposed model uses multiple regularization parameters and therefore can individually control the degree of regularization in each local area of the data space. Several experiments show that the proposed GTM provides better visualization than the conventional GTM approaches.

Visualizing states in autoregressive hidden Markov models using generative topographic mapping

The generative topographic mapping (GTM) algorithm was proposed as a probabilistic re-formulation of the self-organizing map (SOM). The GTM algorithm captures the structure of data by modeling the data with a nonlinear transformation from low-dimensional latent variable space to multidimensional data space, and which can be used as a visualization tool. The object of this paper is to extend the GTM algorithm to deal with multivariate time series. The standard GTM algorithm assumes that the data are independent and identically distributed samples. However, the i.i.d. assumption is clearly inappropriate for time series. In this paper we propose the extension of the GTM for multivariate time series, which we call GTM-ARHMM, by assuming that the time series is generated by autoregressive hidden Markov models (ARHMMs).

Development of an evaluation system for upper limb function using AR technology

This paper develops a prototype system for evaluating upper limb function that combines Internet of Things (IoT) and Augmented Reality (AR) technology. IoT builds the network of patients, test environment and doctors surgery from which the system gathers and exchanges data such as the speeds and locations of hand movement. With the help of AR technology, the real-world test environment is overlaid with the information (e.g. the instructions from doctors, the progress of evaluation) gathered from the IoT. Compared to the conventional system, the detailed information of hand movement supports further assessments and the instructions generated in the AR scene for patients relieve the burden of doctors. The control experiments were conducted to explore the effects of the object size, the existence of obstacles and the hand dominance on the upper limb function using the developed system. The results verified the validity of the developed system.

Visualizing regression data by supervised Generative Topographic Mapping

Generative Topographic Mapping (GTM) is a nonlinear latent variable model introduced by Bishop et al. as a data visualization technique. In this paper, we propose a supervised GTM model and a semi-supervised GTM model. Conventional supervised GTM models use discrete class labels in classification problems, and therefore cannot directly handle continuous output labels in regression problems. To overcome the problem, we propose a supervised GTM model which can naturally handle continuous output labels in regression problems. In order to handle missing labels, we also propose a semi-supervised GTM model that uses both labeled and unlabeled data.

GTM with latent variable dependent length-scale and variance

Generative Topographic Mapping (GTM) is a data visualization technique that uses a nonlinear topographically preserving mapping from latent to data space. Conventional GTM models can be interpreted as a probabilistic model using Gaussian process prior, and therefore the choice of covariance function in the Gaussian process prior has an important effect on the performance. However the conventional GTM models use a covariance function with a constant length-scale for the whole latent space, and therefore fail to adapt to variable smoothness in the nonlinear topographically preserving mapping. In this paper, we propose GTM with latent variable dependent length-scale (GTM-LDLV), which can adjust the smoothness in local areas of the latent space individually.

Combining pairwise coupling classifiers using individual logistic regressions

Pairwise coupling is a popular multi-class classification approach that prepares binary classifiers separating each pair of classes, and then combines the binary classifiers together. This paper proposes a pairwise coupling combination strategy using individual logistic regressions (ILR-PWC). We show analytically and experimentally that the ILR-PWC approach is more accurate than the individual logistic regressions.