Vasile Palade

An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics

Training classifiers with datasets which suffer of imbalanced class distributions is an important problem in data mining. This issue occurs when the number of examples representing the class of interest is much lower than the ones of the other classes. Its presence in many real-world applications has brought along a growth of attention from researchers. We shortly review the many issues in machine learning and applications of this problem, by introducing the characteristics of the imbalanced dataset scenario in classification, presenting the specific metrics for evaluating performance in class imbalanced learning and enumerating the proposed solutions. In particular, we will describe preprocessing, costsensitive learning and ensemble techniques, carrying out an experimental study to contrast these approaches in an intra and inter-family comparison. We will carry out a thorough discussion on the main issues related to using data intrinsic characteristics in this classification problem. This will help to improve the current models with respect to: the presence of small disjuncts, the lack of density in the training data, the overlapping between classes, the identification of noisy data, the significance of the borderline instances, and the dataset shift between the training and the test distributions. Finally, we introduce several approaches and recommendations to address these problems in conjunction with imbalanced data, and we will show some experimental examples on the behavior of the learning algorithms on data with such intrinsic characteristics.

microPred: effective classification of pre-miRNAs for human miRNA gene prediction

MOTIVATION In this article, we show that the classification of human precursor microRNA (pre-miRNAs) hairpins from both genome pseudo hairpins and other non-coding RNAs (ncRNAs) is a common and essential requirement for both comparative and non-comparative computational recognition of human miRNA genes. However, the existing computational methods do not address this issue completely or successfully. Here we present the development of an effective classifier system (named as microPred) for this classification problem by using appropriate machine learning techniques. Our approach includes the introduction of more representative datasets, extraction of new biologically relevant features, feature selection, handling of class imbalance problem in the datasets and extensive classifier performance evaluation via systematic cross-validation methods. RESULTS Our microPred classifier yielded higher and, especially, much more reliable classification results in terms of both sensitivity (90.02%) and specificity (97.28%) than the exiting pre-miRNA classification methods. When validated with 6095 non-human animal pre-miRNAs and 139 virus pre-miRNAs from miRBase, microPred resulted in 92.71% (5651/6095) and 94.24% (131/139) recognition rates, respectively.

FSVM-CIL: Fuzzy Support Vector Machines for Class Imbalance Learning

Support vector machines (SVMs) is a popular machine learning technique, which works effectively with balanced datasets. However, when it comes to imbalanced datasets, SVMs produce suboptimal classification models. On the other hand, the SVM algorithm is sensitive to outliers and noise present in the datasets. Therefore, although the existing class imbalance learning (CIL) methods can make SVMs less sensitive to class imbalance, they can still suffer from the problem of outliers and noise. Fuzzy SVMs (FSVMs) is a variant of the SVM algorithm, which has been proposed to handle the problem of outliers and noise. In FSVMs, training examples are assigned different fuzzy-membership values based on their importance, and these membership values are incorporated into the SVM learning algorithm to make it less sensitive to outliers and noise. However, like the normal SVM algorithm, FSVMs can also suffer from the problem of class imbalance. In this paper, we present a method to improve FSVMs for CIL (called FSVM-CIL), which can be used to handle the class imbalance problem in the presence of outliers and noise. We thoroughly evaluated the proposed FSVM-CIL method on ten real-world imbalanced datasets and compared its performance with five existing CIL methods, which are available for normal SVM training. Based on the overall results, we can conclude that the proposed FSVM-CIL method is a very effective method for CIL, especially in the presence of outliers and noise in datasets.

Quantum-behaved particle swarm optimization: Analysis of individual particle behavior and parameter selection

Quantum-behaved particle swarm optimization (QPSO), motivated by concepts from quantum mechanics and particle swarm optimization (PSO), is a probabilistic optimization algorithm belonging to the bare-bones PSO family. Although it has been shown to perform well in finding the optimal solutions for many optimization problems, there has so far been little analysis on how it works in detail. This paper presents a comprehensive analysis of the QPSO algorithm. In the theoretical analysis, we analyze the behavior of a single particle in QPSO in terms of probability measure. Since the particles behavior is influenced by the contraction-expansion (CE) coefficient, which is the most important parameter of the algorithm, the goal of the theoretical analysis is to find out the upper bound of the CE coefficient, within which the value of the CE coefficient selected can guarantee the convergence or boundedness of the particles position. In the experimental analysis, the theoretical results are first validated by stochastic simulations for the particles behavior. Then, based on the derived upper bound of the CE coefficient, we perform empirical studies on a suite of well-known benchmark functions to show how to control and select the value of the CE coefficient, in order to obtain generally good algorithmic performance in real world applications. Finally, a further performance comparison between QPSO and other variants of PSO on the benchmarks is made to show the efficiency of the QPSO algorithm with the proposed parameter control and selection methods.

Computational Intelligence in Fault Diagnosis

This book presents the most recent concerns and research results in industrial fault diagnosis using intelligent techniques. It focuses on computational intelligence applications to fault diagnosis with real-world applications used in different chapters to validate the different diagnosis methods. The book includes one chapter dealing with a novel coherent fault diagnosis distributed methodology for complex systems.

Neuro-Fuzzy Ensemble Approach for Microarray Cancer Gene Expression Data Analysis

A neuro-fuzzy ensemble model (NFE) is proposed in this paper for analysing the gene expression data from microarray experiments. The proposed approach was tested on three benchmark cancer gene expression data sets. Experimental results show that our NFE model can be used as an efficient computational tool for microarray data analysis. In addition, compared to some most widely used approaches, neuro-fuzzy (NF)-based models not only supply good classification results, but their behavior can also be explained and interpreted in human understandable terms, which provides the researchers with a better understanding of the data

Model-based fault detection and isolation of a steam generator using neuro-fuzzy networks

This paper presents a neuro-fuzzy (NF) networks based scheme for fault detection and isolation (FDI) of a U-tube steam generator (UTSG) in a nuclear power plant. Two types of NF networks are used. A NF based learning and adaptation of Takagi-Sugeno (TS) fuzzy models is used for residual generation, while for residual evaluation a NF network for Mamdani models is used. The NF network for Takagi-Sugeno models is trained with data collected from a full scale UTSG simulator and is used for generating residuals in the fault detection step. A locally linear neuro-fuzzy (LLNF) model is used in the identification of the steam generator. This model is trained using the locally linear model tree (LOLIMOT) algorithm. In the fault isolation part, genetic algorithms are employed to train a Mamdani type NF network, which is used to classify the residuals and take the appropriate decision regarding the actual behavior of the process. Furthermore, a qualitative description of faults is then extracted from the fuzzy rules obtained from the Mamdani NF network. Experimental results presented in the final part of the paper confirm the effectiveness of this approach.

A neural network based multi-classifier system for gene identification in DNA sequences

The paper presents a neural network based multi-classifier system for the identification of Escherichia coli promoter sequences in strings of DNA. As each gene in DNA is preceded by a promoter sequence, the successful location of an E. coli promoter leads to the identification of the corresponding E. coli gene in the DNA sequence. A set of 324 known E. coli promoters and a set of 429 known non-promoter sequences were encoded using four different encoding methods. The encoded sequences were then used to train four different neural networks. The classification results of the four individual neural networks were then combined through an aggregation function, which used a variation of the logarithmic opinion pool method. The weights of this function were determined by a genetic algorithm. The multi-classifier system was then tested on 159 known promoter sequences and 171 non-promoter sequences not contained in the training set. The results obtained through this study proved that the same data set, when presented to neural networks in different forms, can provide slightly varying results. It also proves that when different opinions of more classifiers on the same input data are integrated within a multi-classifier system, we can obtain results that are better than the individual performances of the neural networks. The performances of our multi-classifier system outperform the results of other prediction systems for E. coli promoters developed so far.

Fault diagnosis of an industrial gas turbine using neuro−fuzzy methods

Abstract The paper focuses on the application of neuro-fuzzy techniques in fault detection and isolation. The objective of this paper is to detect and isolate faults to an industrial gas turbine, with emphasis on faults occurred in the actuator part of the gas turbine. A neuro-fuzzy based learning and adaptation of TSK fuzzy models is used for residual generation, while for residual evaluation a neuro-fuzzy classifier for Mamdani models is used. The paper is concerned on how to obtain an interpretable fault classifier as well as interpretable models for residual generation.

Towards interactive Machine Learning (iML): Applying Ant Colony Algorithms to Solve the Traveling Salesman Problem with the Human-in-the-Loop Approach

Most Machine Learning (ML) researchers focus on automatic Machine Learning (aML) where great advances have been made, for example, in speech recognition, recommender systems, or autonomous vehicles. Automatic approaches greatly benefit from the availability of “big data”. However, sometimes, for example in health informatics, we are confronted not a small number of data sets or rare events, and with complex problems where aML-approaches fail or deliver unsatisfactory results. Here, interactive Machine Learning (iML) may be of help and the “human-in-the-loop” approach may be beneficial in solving computationally hard problems, where human expertise can help to reduce an exponential search space through heuristics.