Koichiro Yamauchi

Detecting concept drift using statistical testing

Detecting concept drift is important for dealing with realworld online learning problems. To detect concept drift in a small number of examples, methods that have an online classifier and monitor its prediction errors during the learning have been developed. We have developed such a detection method that uses a statistical test of equal proportions. Experimental results showed that our method performed well in detecting the concept drift in five synthetic datasets that contained various types of concept drift.

Incremental learning methods with retrieving of interfered patterns

There are many cases that a neural-network-based system must memorize some new patterns incrementally. However, if the network learns the new patterns only by referring to them, it probably forgets old memorized patterns, since parameters in the network usually correlate not only to the old memories but also to the new patterns. A certain way to avoid the loss of memories is to learn the new patterns with all memorized patterns. It needs, however, a large computational power. To solve this problem, we propose incremental learning methods with retrieving interfered patterns (ILRI). In these methods, the system employs a modified version of a resource allocating network (RAN) which is one variation of a generalized radial basis function (GRBF). In ILRI, the RAN learns new patterns with a relearning of a few number of retrieved past patterns that are interfered with the incremental learning. In this paper, we construct ILRI in two steps. In the first step, we construct a system which searches the interfered patterns from past input patterns stored in a database. In the second step, we improve the first system in such a way that the system does not need the database. In this case, the system regenerates the input patterns approximately in a random manner. The simulation results show that these two systems have almost the same ability, and the generalization ability is higher than other similar systems using neural networks and k-nearest neighbors.

ACE: adaptive classifiers-ensemble system for concept-drifting environments

Most machine learning algorithms assume stationary environments, require a large number of training examples in advance, and begin the learning from scratch. In contrast, humans learn in changing environments with sequential training examples and leverage prior knowledge in new situations. To deal with real-world problems in changing environments, the ability to make human-like quick responses must be developed in machines. Many researchers have presented learning systems that assume the presence of hidden context and concept drift. In particular, several systems have been proposed that use ensembles of classifiers on sequential chunks of training examples. These systems can respond to gradual changes in large-scale data streams but have problems responding to sudden changes and leveraging prior knowledge of recurring contexts. Moreover, these are not pure online learning systems. We propose an online learning system that uses an ensemble of classifiers suited to recent training examples. We use experiments to show that this system can leverage prior knowledge of recurring contexts and is robust against various noise levels and types of drift.

Adaptive Classifiers-Ensemble System for Tracking Concept Drift

Adapting to various types of concept drift is important for dealing with real-world online learning problems. To achieve this, we previously reported an online learning system that uses an ensemble of classifiers, the adaptive classifiers-ensemble (ACE) system. ACE consists of one online classifier, many batch classifiers, and a drift detection mechanism. To improve the performance of ACE, we have improved the weighting method, which combines the outputs of classifiers, and have added a new classifier pruning method. Experimental results showed that the enhanced ACE performed well for a synthetic dataset that contained both sudden and gradual changes and recurring concepts.

Learning, detecting, understanding, and predicting concept changes

The demand for learning machines that can adapt to concept change, the change over time of the statistical properties of a target variable, has become more urgent. We, therefore, propose a system in which multiple online and offline classifiers are used for learning changing concepts. Our system is able to: respond to both sudden and gradual changes, handle recurring concepts, detect the occurrence of change, understand the hidden contexts of past concepts, and predict the next concept. We evaluate the effectiveness of our systems elements and demonstrate that our system performed well with synthetic concept-drifting and concept-shifting datasets.

A self-supervised learning system for pattern recognition by sensory integration

Artificial neural networks are useful tools for pattern recognition because they realize nonlinear mapping between input and output spaces. This ability is tuned by supervised learning methods such as back-propagation. In the supervised learning methods, desired outputs of the neural network are needed. However, the desired outputs are usually unknown in unpredictable environments. To solve this problem, this paper presents a self-supervised learning system for category detection. This system learns categories of objects automatically by integrating information from several sensors. We assume that these sensory inputs are always ambiguous patterns that include some noises according to deformations of the objects. After the learning, the system recognizes objects, also controlling the priority of each sensor, according to the deformation of the sensory input pattern.In the simulation, the system is applied to several learning and recognition tasks using artificial or actual sensory inputs. In all tasks, the system found the categories. Particularly, we applied the new system to the learning of five Japanese vowels with the corresponding shapes of the mouth. As result, the system became to yield specific outputs corresponding to each vowel.

Incremental Learning on a Budget and its Application to Quick Maximum Power Point Tracking of Photovoltaic Systems

Maximum power point tracking(MPPT) is an essential technique to improving the efficiency of renewable energy systems. Although various techniques exist that can realize MPPT, there are fewer techniques that can realize quick control using conventional circuit design. In this paper, we propose a quick MPPT converter using a limited general regression neural network (LGRNN). The proposed LGRNN is an incremental learning method for regression on a budget [1]. Therefore, the LGRNN is able to work on small embedded systems, which allows the MPPT converter to be constructed at low cost using the normal combination of a chopper circuit and a controlling microcomputer. This means that the MPPT converter can be constructed in a low cost. The LGRNN learns the maximum power point (MPP) found by the perturb and observe (P&O) method, and sets the reference voltage of the converter immediately after a sudden irradiation change. By using this strategy, the MPPT quickly without predetermination of parameters. The experimental results suggests that after learning, the proposed converter controls the chopper circuit within about 20 [ms] after sudden irradiation changes. Moreover, the converter was designed to be attached to each solar panel to obtain the MPP of each panel. The proposed converter was tested with two series-connected solar panels. The results shows that the proposed system maintains a high efficiency even if one of the two panels is shadowed.

Pruning with replacement and automatic distance metric detection in limited general regression neural networks

In this paper, we propose a limited general regression neural network (LGRNN) for embedded systems. The LGRNN is an improved version of general regression neural network that continues incremental learning under a fixed number of hidden units.

Detecting sudden concept drift with knowledge of human behavior

Concept drift, the change over time of the statistical properties of the target variable, is a serious problem for online learning systems. To overcome this problem, we propose a method inspired by human behavior for detecting sudden concept drift. We first conducted a human behavior experiment to investigate our working hypothesis that humans can detect changes quickly when their confident classifications are rejected despite the fact that their recent classification accuracy is high. The human behavior experiments supported our working hypothesis. We then have proposed the leaky integrate-and-detect (LID) model based on our working hypothesis. Our computer experiments showed LID was able to detect sudden changes quickly and accurately in an environment that includes noise and gradual changes.

Color coordination system on case based reasoning system using neural network

We propose a case-based reasoning (CBR) system whose case database consists of a neural network, and describe its application to a color coordination system. Furthermore, we apply a neural network to the CBR system to deal with fuzzy values which represent colors. The neural network in the CBR system, however, must learn a new case incrementally without forgetting the learned instances. To realize this ability, we use a new incremental learning method proposed by the us to reduce the computational complexity for the learning. In the new method, the system learns the generalized radial cases function both the new case and some old cases that are predicted and being interfered by the learning. For the experiments, we constructed a color coordination system for a make-up around eyes. The result gave appropriate combinations of colors that satisfied the user.