Jacek M. Zurada

Normalized Mutual Information Feature Selection

A filter method of feature selection based on mutual information, called normalized mutual information feature selection (NMIFS), is presented. NMIFS is an enhancement over Battitis MIFS, MIFS-U, and mRMR methods. The average normalized mutual information is proposed as a measure of redundancy among features. NMIFS outperformed MIFS, MIFS-U, and mRMR on several artificial and benchmark data sets without requiring a user-defined parameter. In addition, NMIFS is combined with a genetic algorithm to form a hybrid filter/wrapper method called GAMIFS. This includes an initialization procedure and a mutation operator based on NMIFS to speed up the convergence of the genetic algorithm. GAMIFS overcomes the limitations of incremental search algorithms that are unable to find dependencies between groups of features.

Artificial Intelligence and Soft Computing

In recent years wind energy has been the fastest growing branch of the power generation industry. Maintenance of the wind turbine generates its the largest cost. A remote monitoring is a common method to reduce this cost. Growing number of monitored turbines requires an automatized way of support for diagnostic experts. Early fault detection and identification is still a very challenging task. A tool, which can alert an engineer about potentially dangerous cases, is required to work in real-time. The goal of this paper is to show an efficient system to online classification of operational states of the wind turbines and to detecting their early fault cases. The proposed system was designed as a hybrid of ART-2 and RBF networks. It had been proved before that the ART-type ANNs can successfully recognize operational states of a wind turbine during the diagnostic process. There are some difficulties, however, when classification is done in real-time. The disadvantages of using a classic ART-2 network are pointed and it is explained why the RBF unit of the hybrid system is needed to have a proper classification of turbine operational states.

An approach to multimodal biomedical image registration utilizing particle swarm optimization

Biomedical image registration, or geometric alignment of two-dimensional and/or three-dimensional (3D) image data, is becoming increasingly important in diagnosis, treatment planning, functional studies, computer-guided therapies, and in biomedical research. Registration based on intensity values usually requires optimization of some similarity metric between the images. Local optimization techniques frequently fail because functions of these metrics with respect to transformation parameters are generally nonconvex and irregular and, therefore, global methods are often required. In this paper, a new evolutionary approach, particle swarm optimization, is adapted for single-slice 3D-to-3D biomedical image registration. A new hybrid particle swarm technique is proposed that incorporates initial user guidance. Multimodal registrations with initial orientations far from the ground truth were performed on three volumes from different modalities. Results of optimizing the normalized mutual information similarity metric were compared with various evolutionary strategies. The hybrid particle swarm technique produced more accurate registrations than the evolutionary strategies in many cases, with comparable convergence. These results demonstrate that particle swarm approaches, along with evolutionary techniques and local methods, are useful in image registration, and emphasize the need for hybrid approaches for difficult registration problems.

Complex-valued multistate neural associative memory

A model of a multivalued associative memory is presented. This memory has the form of a fully connected attractor neural network composed of multistate complex-valued neurons. Such a network is able to perform the task of storing and recalling gray-scale images. It is also shown that the complex-valued fully connected neural network may be considered as a generalization of a Hopfield network containing real-valued neurons. A computational energy function is introduced and evaluated in order to prove network stability for asynchronous dynamics. Storage capacity as related to the number of accessible neuron states is also estimated.

Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance.

This study investigates the effect of class imbalance in training data when developing neural network classifiers for computer-aided medical diagnosis. The investigation is performed in the presence of other characteristics that are typical among medical data, namely small training sample size, large number of features, and correlations between features. Two methods of neural network training are explored: classical backpropagation (BP) and particle swarm optimization (PSO) with clinically relevant training criteria. An experimental study is performed using simulated data and the conclusions are further validated on real clinical data for breast cancer diagnosis. The results show that classifier performance deteriorates with even modest class imbalance in the training data. Further, it is shown that BP is generally preferable over PSO for imbalanced training data especially with small data sample and large number of features. Finally, it is shown that there is no clear preference between oversampling and no compensation approach and some guidance is provided regarding a proper selection.

Swarm and Evolutionary Computation

This paper presents a work inspired by the Pachycondyla apicalis ants behavior for the clustering problem. These ants have a simple but efficient prey search strategy: when they capture their prey, they return straight to their nest, drop off the prey and systematically return back to their original position. This behavior has already been applied to optimization, as the API meta-heuristic. API is a shortage of api-calis. Here, we combine API with the ability of ants to sort and cluster. We provide a comparison against Ant clustering Algorithm and K-Means using Machine Learning repository datasets. API introduces new concepts to ant-based models and gives us promising results.

Computational Intelligence Methods For Rule-Based Data Understanding

In many applications, black-box prediction is not satisfactory, and understanding the data is of critical importance. Typically, approaches useful for understanding of data involve logical rules, evaluate similarity to prototypes, or are based on visualization or graphical methods. This paper is focused on the extraction and use of logical rules for data understanding. All aspects of rule generation, optimization, and application are described, including the problem of finding good symbolic descriptors for continuous data, tradeoffs between accuracy and simplicity at the rule-extraction stage, and tradeoffs between rejection and error level at the rule optimization stage. Stability of rule-based description, calculation of probabilities from rules, and other related issues are also discussed. Major approaches to extraction of logical rules based on neural networks, decision trees, machine learning, and statistical methods are introduced. Optimization and application issues for sets of logical rules are described. Applications of such methods to benchmark and real-life problems are reported and illustrated with simple logical rules for many datasets. Challenges and new directions for research are outlined.

Extraction of rules from artificial neural networks for nonlinear regression

Neural networks (NNs) have been successfully applied to solve a variety of application problems including classification and function approximation. They are especially useful as function approximators because they do not require prior knowledge of the input data distribution and they have been shown to be universal approximators. In many applications, it is desirable to extract knowledge that can explain how Me problems are solved by the networks. Most existing approaches have focused on extracting symbolic rules for classification. Few methods have been devised to extract rules from trained NNs for regression. This article presents an approach for extracting rules from trained NNs for regression. Each rule in the extracted rule set corresponds to a subregion of the input space and a linear function involving the relevant input attributes of the data approximates the network output for all data samples in this subregion. Extensive experimental results on 32 benchmark data sets demonstrate the effectiveness of the proposed approach in generating accurate regression rules.

A new design method for the complex-valued multistate Hopfield associative memory

A method to store each element of an integral memory set M subset {1,2,...,K}/sup n/ as a fixed point into a complex-valued multistate Hopfield network is introduced. The method employs a set of inequalities to render each memory pattern as a strict local minimum of a quadratic energy landscape. Based on the solution of this system, it gives a recurrent network of n multistate neurons with complex and symmetric synaptic weights, which operates on the finite state space {1,2,...,K}/sup n/ to minimize this quadratic functional. Maximum number of integral vectors that can be embedded into the energy landscape of the network by this method is investigated by computer experiments. This paper also enlightens the performance of the proposed method in reconstructing noisy gray-scale images.

Nonlinear blind source separation using a radial basis function network

This paper proposes a novel neural-network approach to blind source separation in nonlinear mixture. The approach utilizes a radial basis function (RBF) neural-network to approximate the inverse of the nonlinear mixing mapping which is assumed to exist and able to be approximated using an RBF network. A contrast function which consists of the mutual information and partial moments of the outputs of the separation system, is defined to separate the nonlinear mixture. The minimization of the contrast function results in the independence of the outputs with desirable moments such that the original sources are separated properly. Two learning algorithms for the parametric RBF network are developed by using the stochastic gradient descent method and an unsupervised clustering method. By virtue of the RBF neural network, this proposed approach takes advantage of high learning convergence rate of weights in the hidden layer and output layer, natural unsupervised learning characteristics, modular structure, and universal approximation capability. Simulation results are presented to demonstrate the feasibility, robustness, and computability of the proposed method.