Khalid M. Salama

Utilizing multiple pheromones in an ant-based algorithm for continuous-attribute classification rule discovery

The cAnt-Miner algorithm is an Ant Colony Optimization (ACO) based technique for classification rule discovery in problem domains which include continuous attributes. In this paper, we propose several extensions to cAnt-Miner. The main extension is based on the use of multiple pheromone types, one for each class value to be predicted. In the proposed @mcAnt-Miner algorithm, an ant first selects a class value to be the consequent of a rule and the terms in the antecedent are selected based on the pheromone levels of the selected class value; pheromone update occurs on the corresponding pheromone type of the class value. The pre-selection of a class value also allows the use of more precise measures for the heuristic function and the dynamic discretization of continuous attributes, and further allows for the use of a rule quality measure that directly takes into account the confidence of the rule. Experimental results on 20 benchmark datasets show that our proposed extension improves classification accuracy to a statistically significant extent compared to cAnt-Miner, and has classification accuracy similar to the well-known Ripper and PART rule induction algorithms.

Learning Bayesian network classifiers using ant colony optimization

Bayesian networks are knowledge representation tools that model the (in)dependency relationships among variables for probabilistic reasoning. Classification with Bayesian networks aims to compute the class with the highest probability given a case. This special kind is referred to as Bayesian network classifiers. Since learning the Bayesian network structure from a dataset can be viewed as an optimization problem, heuristic search algorithms may be applied to build high-quality networks in medium- or large-scale problems, as exhaustive search is often feasible only for small problems. In this paper, we present our new algorithm, ABC-Miner, and propose several extensions to it. ABC-Miner uses ant colony optimization for learning the structure of Bayesian network classifiers. We report extended computational results comparing the performance of our algorithm with eight other classification algorithms, namely six variations of well-known Bayesian network classifiers, cAnt-Miner for discovering classification rules and a support vector machine algorithm.

Learning neural network structures with ant colony algorithms

Ant colony optimization (ACO) has been successfully applied to classification, where the aim is to build a model that captures the relationships between the input attributes and the target class in a given domain’s dataset. The constructed classification model can then be used to predict the unknown class of a new pattern. While artificial neural networks are one of the most widely used models for pattern classification, their application is commonly restricted to fully connected three-layer topologies. In this paper, we present a new algorithm, ANN-Miner, which uses ACO to learn the structure of feed-forward neural networks. We report computational results on 40 benchmark datasets for several variations of the algorithm. Performance is compared to the standard three-layer structure trained with two different weight-learning algorithms (back propagation, and the

A Novel Ant Colony Algorithm for Building Neural Network Topologies

\hbox {ACO}_{\mathbb {R}}

Clustering-based Bayesian Multi-net Classifier construction with Ant Colony Optimization

ACOR algorithm), and also to a greedy algorithm for learning NN structures. A nonparametric Friedman test is used to determine statistical significance. In addition, we compare our proposed algorithm with NEAT, a prominent evolutionary algorithm for evolving neural networks, as well as three different well-known state-of-the-art classifiers, namely the C4.5 decision tree induction algorithm, the Ripper classification rule induction algorithm, and support vector machines.

Exploring different rule quality evaluation functions in ACO-based classification algorithms

Bayesian Multi-nets BMNs are a special kind of Bayesian network BN classifiers that consist of several local Bayesian networks, one for each predictable class, to model an asymmetric set of variable dependencies given each class value. Deterministic methods using greedy local search are the most frequently used methods for learning the structure of BMNs based on optimizing a scoring function. Ant Colony Optimization ACO is a meta-heuristic global search method for solving combinatorial optimization problems, inspired by the behavior of real ant colonies. In this paper, we propose two novel ACO-based algorithms with two different approaches to build BMN classifiers: ABC-Miner^{mn}_l and ABC-Miner^{mn}_g. The former uses a local learning approach, in which the ACO algorithm completes the construction of one local BN at a time. The latter uses a global approach, which involves building a complete BMN classifier by each single ant in the colony. We experimentally evaluate the performance of our ant-based algorithms on 33 benchmark classification datasets, where our proposed algorithms are shown to be significantly better than other commonly used deterministic algorithms for learning various Bayesian classifiers in the literature, as well as competitive to other well-known classification algorithms.

Classification with cluster-based Bayesian multi-nets using Ant Colony Optimisation

A re-occurring challenge in applying feed-forward neural networks to a new dataset is the need to manually tune the neural network topology. If one’s attention is restricted to fully-connected three-layer networks, then there is only the need to manually tune the number of neurons in the single hidden layer. In this paper, we present a novel Ant Colony Optimization (ACO) algorithm that optimizes neural network topology for a given dataset. Our algorithm is not restricted to three-layer networks, and can produce topologies that contain multiple hidden layers, and topologies that do not have full connectivity between successive layers. Our algorithm uses Backward Error Propagation (BP) as a subroutine, but it is possible, in general, to use any neural network learning algorithm within our ACO approach instead. We describe all the elements necessary to tackle our learning problem using ACO, and experimentally compare the classification performance of the optimized topologies produced by our ACO algorithm with the standard fully-connected three-layer network topology most-commonly used in the literature.

Extending the ABC-Miner Bayesian Classification Algorithm

Ant-Miner is an ant-based algorithm for the discovery of classification rules. This paper proposes four extensions to Ant-Miner: 1) we allow the use of a logical negation operator in the antecedents of constructed rules; 2) we use stubborn ants, an ACO-variation in which an ant is allowed to take into consideration its own personal past history; 3) we use multiple types of pheromone, one for each permitted rule class, i.e. an ant would first select the rule class and then deposit the corresponding type of pheromone; 4) we allow each ant to have its own value of the α and β parameters, which in a sense means that each ant has its own individual personality. Empirical results show improvements in the algorithms performance in terms of the simplicity of the generated rule set, the number of trials, and the predictive accuracy.