Ah-Hwee Tan

A fast pruned-extreme learning machine for classification problem

Extreme learning machine (ELM) represents one of the recent successful approaches in machine learning, particularly for performing pattern classification. One key strength of ELM is the significantly low computational time required for training new classifiers since the weights of the hidden and output nodes are randomly chosen and analytically determined, respectively. In this paper, we address the architectural design of the ELM classifier network, since too few/many hidden nodes employed would lead to underfitting/overfitting issues in pattern classification. In particular, we describe the proposed pruned-ELM (P-ELM) algorithm as a systematic and automated approach for designing ELM classifier network. P-ELM uses statistical methods to measure the relevance of hidden nodes. Beginning from an initial large number of hidden nodes, irrelevant nodes are then pruned by considering their relevance to the class labels. As a result, the architectural design of ELM network classifier can be automated. Empirical study of P-ELM on several commonly used classification benchmark problems and with diverse forms of hidden node functions show that the proposed approach leads to compact network classifiers that generate fast response and robust prediction accuracy on unseen data, comparing with traditional ELM and other popular machine learning approaches.

Rule extraction: from neural architecture to symbolic representation

This paper shows how knowledge, in the form of fuzzy rules, can be derived from a supervised learning neural network called fuzzy ARTMAP. Rule extraction proceeds in two stages: pruning, which simplifies the network structure by removing excessive recognition categories and weights; and quantization of continuous learned weights, which allows the final system state to be translated into a usable set of descriptive rules. Three benchmark studies illustrate the rule extraction methods: (1) Pima Indian diabetes diagnosis, (2) mushroom classification and (3) DNA promoter recognition. Fuzzy ARTMAP and ART-EMAP are compared with the ADAP algorithm, the k nearest neighbor system, the back-propagation network and the C4.5 decision tree. The ARTMAP rule extraction procedure is also compared with the Knowledgetron and NOFM algorithms, which extract rules from back-propagation networks. Simulation results consistently indicate that ARTMAP rule extraction produces compact sets of comprehensible rules for which accura...

On Quantitative Evaluation of Clustering Systems

Clustering refers to the task of partitioning unlabelled data into meaningful groups (clusters). It is a useful approach in data mining processes for identifying hidden patterns and revealing underlying knowledge from large data collections. The application areas of clustering, to name a few, include image segmentation, information retrieval, document classification, associate rule mining, web usage tracking, and transaction analysis.

Adaptive resonance associative map

Abstract This article introduces a neural architecture termed Adaptive Resonance Associative Map (ARAM) that extends unsupervised Adaptive Resonance Theory (ART) systems for rapid, yet stable, heteroassociative learning. ARAM can be visualized as two overlapping ART networks sharing a single category field. Although ARAM is simpler in architecture than another class of supervised ART models known as ARTMAP, it produces classification performance equivalent to that of ARTMAP. As ARAM network structure and operations are symmetrical, associative recall can be performed in both directions. With maximal vigilance settings, ARAM encodes pattern pairs explicitly as cognitive chunks and thus guarantees perfect storage and recall of an arbitrary number of arbitrary pattern pairs. Simulations on an iris plant and a sonar return recognition problems compare ARAM classification performance with that of counterpropagation network, K-nearest neighbor system, and back propagation network. Associative recall experiments on two pattern sets show that, besides the advantages of fast learning, guaranteed perfect storage, and full memory capacity, ARAM produces a stronger noise immunity than Bidirectional Associative Memory (BAM).

Integrating Temporal Difference Methods and Self-Organizing Neural Networks for Reinforcement Learning With Delayed Evaluative Feedback

This paper presents a neural architecture for learning category nodes encoding mappings across multimodal patterns involving sensory inputs, actions, and rewards. By integrating adaptive resonance theory (ART) and temporal difference (TD) methods, the proposed neural model, called TD fusion architecture for learning, cognition, and navigation (TD-FALCON), enables an autonomous agent to adapt and function in a dynamic environment with immediate as well as delayed evaluative feedback (reinforcement) signals. TD-FALCON learns the value functions of the state-action space estimated through on-policy and off-policy TD learning methods, specifically state-action-reward-state-action (SARSA) and Q-learning. The learned value functions are then used to determine the optimal actions based on an action selection policy. We have developed TD-FALCON systems using various TD learning strategies and compared their performance in terms of task completion, learning speed, as well as time and space efficiency. Experiments based on a minefield navigation task have shown that TD-FALCON systems are able to learn effectively with both immediate and delayed reinforcement and achieve a stable performance in a pace much faster than those of standard gradient-descent-based reinforcement learning systems.

Cascade ARTMAP: integrating neural computation and symbolic knowledge processing

This paper introduces a hybrid system termed cascade adaptive resonance theory mapping (ARTMAP) that incorporates symbolic knowledge into neural-network learning and recognition. Cascade ARTMAP, a generalization of fuzzy ARTMAP, represents intermediate attributes and rule cascades of rule-based knowledge explicitly and performs multistep inferencing. A rule insertion algorithm translates if-then symbolic rules into cascade ARTMAP architecture. Besides that initializing networks with prior knowledge can improve predictive accuracy and learning efficiency, the inserted symbolic knowledge can be refined and enhanced by the cascade ARTMAP learning algorithm. By preserving symbolic rule form during learning, the rules extracted from cascade ARTMAP can be compared directly with the originally inserted rules. Simulations on an animal identification problem indicate that a priori symbolic knowledge always improves system performance, especially with a small training set. Benchmark study on a DNA promoter recognition problem shows that with the added advantage of fast learning, cascade ARTMAP rule insertion and refinement algorithms produce performance superior to those of other machine learning systems and an alternative hybrid system known as knowledge-based artificial neural network (KBANN). Also, the rules extracted from cascade ARTMAP are more accurate and much cleaner than the NofM rules extracted from KBANN.

Modelling situation awareness for Context-aware Decision Support

Situation awareness modelling is popularly used in the command and control domain for situation assessment and decision support. However, situation models in real-world applications are typically complex and not easy to use. This paper presents a Context-aware Decision Support (CaDS) system, which consists of a situation model for shared situation awareness modelling and a group of entity agents, one for each individual user, for focused and customized decision support. By incorporating a rule-based inference engine, the entity agents provide functions including event classification, action recommendation, and proactive decision making. The implementation and the performance of the proposed system are demonstrated through a case study on a simulated command and control application.

Topic Detection, Tracking, and Trend Analysis Using Self-Organizing Neural Networks

We address the problem of Topic Detection and Tracking (TDT) and subsequently detecting trends from a stream of text documents. Formulating TDT as a clustering problem in a class of self-organizing neural networks, we propose an incremental clustering algorithm. On this setup we show how trends can be identified. Through experimental studies, we observe that our method enables discovering interesting trends that are deducible only from reading all relevant documents.

Intelligence Through Interaction: Towards a Unified Theory for Learning

Machine learning, a cornerstone of intelligent systems, has typically been studied in the context of specific tasks, including clustering (unsupervised learning), classification (supervised learning), and control (reinforcement learning). This paper presents a learning architecture within which a universal adaptation mechanism unifies a rich set of traditionally distinct learning paradigms, including learning by matching, learning by association, learning by instruction, and learning by reinforcement. In accordance with the notion of embodied intelligence, such a learning theory provides a computational account of how an autonomous agent may acquire the knowledge of its environment in a real-time, incremental, and continuous manner. Through a case study on a minefield navigation domain, we illustrate the efficacy of the proposed model, the learning paradigms encompassed, and the various types of knowledge learned.

FALCON: a fusion architecture for learning, cognition, and navigation

This work presents a natural extension of self-organizing neural network architecture for learning cognitive codes across multi-modal patterns involving sensory input, actions, and rewards. The proposed cognitive model, called FALCON, enables an autonomous agent to adapt and function in a dynamic environment. Simulations based on a minefield navigation task indicate that the system is able to adapt amazingly well and learns rapidly through its interaction with the environment in an online and incremental manner. The scalability and robustness of the system is further enhanced by an online code evaluation and pruning procedure, that maintains the number of cognitive codes at a manageable size without degradation of system performance.