Jan Kozak

Ant colony decision trees: a new method for constructing decision trees based on ant colony optimization

In this paper, we would like to propose a new method for constructing decision trees based on Ant Colony Optimization (ACO). The ACO is a metaheuristic inspired by the behavior of real ants, where they search for optimal solutions by considering both local heuristic and previous knowledge, observed by pheromone changes. Good results of the ant colony algorithms for solving combinatorial optimization problems suggest an appropriate effectiveness of the approach also in the task of constructing decision trees. In order to improve the accuracy of decision trees we propose an Ant Colony algorithm for constructing Decision Trees (ACDT). A heuristic function used in a new algorithm is based on the splitting rule of the CART algorithm (Classification and Regression Trees). The proposed algorithm is evaluated on a number of well-known benchmark data sets from the UCI Machine Learning repository. What deserves particular attention is the fact that empirical results clearly show that ACDT performs very good while comparing to other techniques.

An adaptive discretization in the ACDT algorithm for continuous attributes

Decision tree induction has been widely used to generate classifiers from training data through a process of recursively splitting the data space. In the case of training on continuous-valued data, the associated attributes must be discretized in advance or during the learning process. The commonly used method is to partition the attribute range into two or several intervals using single or a set of cut points. One inherent disadvantage in these methods is that the use of sharp cut points makes the induced decision trees sensitive to noise. To overcome this problem this paper presents an alternative method called adaptive discretization based on Ant Colony Decision Tree (ACDT) approach. Experimental results showed that, by using that methodology, better classification accuracy has been obtained in both training and testing data sets in majority of cases concerning the classical decision tree constructed by ants. It suggests that the robustness of decision trees could be improved by means of this approach.

Multiple Boosting in the Ant Colony Decision Forest meta-classifier

The idea of ensemble methodology is to combine multiple predictive models in order to achieve a better prediction performance. In this task we analyze the self-adaptive methods for improving the performance of Ant Colony Decision Tree and Forest algorithms. Our goal is to present and compare new meta-ensemble approaches based on Ant Colony Optimization. The proposed meta-classifiers (consisting of homogeneous classifiers) can be characterized by the self-adaptability or the good accommodation with the analyzed data sets and offer appropriate classification accuracy.In this article we provide an overview of ensemble methods in classification tasks and concentrate on the different methodologies, such as Bagging, Boosting and Random Forest. We present all important types of ensemble methods including Boosting and Bagging in context of distributed approach, where agent-ants create better solutions employing adaptive mechanisms. Self adaptive, combining methods and modeling appropriate issues, such as ensembles presented here are discussed in context of the quality of the results. Smaller trees in decision forest without loss of accuracy are achieved during the analysis of different data sets.

New Algorithms for Generation Decision Trees—Ant-Miner and Its Modifications

In our approach we want to ensure the good performance of Ant- Miner by applying the well-known (from the ACO algorithm) two pheromone updating rules: local and global, and the main pseudo-random proportional rule, which provides appropriate mechanisms for search space: exploitation and exploration. Now we can utilize an improved expression of this classification rule discovery system as an Ant-Colony-Miner. Further modifications are connected with the simplicity of the heuristic function used in the standard Ant-Miner. We propose to employing a new heuristic function based on quantitative, not qualitative parameters used during the classification process. The main transition rule will be changed dynamically as a result of the simple frequency analysis of the number of cases from the point of view characteristic partitions. This simplified heuristic function will be compensated by the pheromone update in different degrees, which helps ants to collaborate and is a good stimulant on ants’ behavior during the rule construction. The comparative study will be conducted using 5 data sets from the UCI Machine Learning repository.

Enhancing the effectiveness of Ant Colony Decision Tree algorithms by co-learning

Graphical abstractDisplay Omitted HighlightsACO techniques provide a way to efficiently search for solutions via colearning.ACO applied to data mining tasks is one of these methods and the focus of this paper..The ACDT approach generates solutions efficiently and effectively.The ACDT approach is tested in the context of the bi-criteria evaluation function.The empirical results clearly show that the ACDT algorithm creates good solutions. Data mining and visualization techniques for high-dimensional data provide helpful information to substantially augment decision-making. Optimization techniques provide a way to efficiently search for these solutions. ACO applied to data mining tasks - a decision tree construction - is one of these methods and the focus of this paper. The Ant Colony Decision Tree (ACDT) approach generates solutions efficiently and effectively but scales poorly to large problems. This article merges the methods that have been developed for better construction of decision trees by ants. The ACDT approach is tested in the context of the bi-criteria evaluation function by focusing on two problems: the size of the decision trees and the accuracy of classification obtained during ACDT performance. This approach is tested in co-learning mechanism, it means agents-ants can interact during the construction decision trees via pheromone values. This cooperation is a chance of getting better results. The proposed methodology of analysis of ACDT is tested in a number of well-known benchmark data sets from the UCI Machine Learning Repository. The empirical results clearly show that the ACDT algorithm creates good solutions which are located in the Pareto front. The software that implements the ACDT algorithm used to generate the results of this study can be downloaded freely from http://www.acdtalgorithm.com.

Collective data mining in the ant colony decision tree approach

We confirm that agent-ants can learn only via the pheromone trail.It is better to use both the heuristic function and the pheromone trail.The ACDT alg. makes it possible to optimize the construction of decision trees based on a heuristic.The pheromone map clearly present a data set as the solution space and pheromone trail values.Analysis of pheromone maps shows that the ACDT alg. searches different subspaces of the solution space. This paper considers the topic of cooperation between agent ants in an Ant Colony Optimization (ACO) algorithm that is used to construct decision trees (Ant Colony Decision Tree or ACDT). To follow a suitable methodology, the paper presents a formal definition of the ACDT algorithm with a focus on the influence that Ant Colony Optimization algorithms have on the obtained results. The aim of this paper is to provide the rationale for using swarm intelligence (i.e., ACO) in the process of constructing decision trees. Many experiments were conducted to provide a solid justification. These experiments tested cooperation between agent ants in ant colony algorithms with different ACO performance scenarios: the application of only a pheromone trail, the application of only a heuristic function, the application of both components, and the application of neither component. Additionally, different values of the pheromone trail were tested at various stages of the algorithms operation and pheromone representations were presented. The experiments were conducted on 30 publicly available data sets; all observations were preceded by statistical tests. This paper confirms that it is reasonable to use the pheromone trail and balanced heuristics. Moreover, we found that, for the ACDT algorithm, good results can also be obtained without heuristics.

An Ant Colony Optimization Algorithm for an Automatic Categorization of Emails

This article presents a new approach to an automatic categorization of email messages which is based on Ant Colony Optimization algorithms (ACO). The aim of this paper is to create an algorithm that would allow one to improve the classification of emails into folders (the email foldering problem) by using solutions that have been applied in Ant Colony algorithms, data mining and Social Network Analysis (SNA). The new algorithm which is proposed here has been tested on the publicly available Enron email data set. The obtained results confirm that this approach allows one to improve the accuracy with which new emails are assigned to particular folders based on an analysis of previous correspondence.

Dynamic Version of the ACDT/ACDF Algorithm for H-Bond Data Set Analysis

This article is devoted to the new application of the ACDT/ACDF algorithms. In this work we distinguish ant colony optimization and join it with decision tree construction algorithms, the proposed approach builds more stable decision forests. Additionally, we would like to mention that it is possible to analyze the overloaded data sets. Several methods are proposed in this study, each considered different pseudo-samples from training data sets. We combine ideas from ACO, Boosting and Random Forests. We show that our algorithms perform comparable to common approaches. Moreover, we demonstrate the suitability of our method to H-bonds detections in protein structures.

New insights of cooperation among ants in Ant Colony Decision Trees

In this paper, we propose a new method for constructing decision trees based on Ant Colony Optimization (ACO). The ACO is a metaheuristic inspired by the behavior of real ants, where they search for optimal solutions by considering both local heuristic and previous knowledge, observed by pheromone changes. Good results of the ant colony algorithms for solving combinatorial optimization problems suggest an appropriate effectiveness of the approach also in the task of constructing decision trees. In order to improve the accuracy of decision trees we propose an Ant Colony algorithm for constructing Decision Trees (ACDT - www.ACDTalgorithm.com). A heuristic function used in the new algorithm is based on the splitting rule of the CART algorithm (Classification and Regression Trees). The proposed algorithm is evaluated in terms of exploration/exploitation rate, heuristic function, cooperation among ants, initial pheromone value.

Ant colony decision forest meta-ensemble

This paper is devoted to the study of an extension of Ant Colony Decision Tree (ACDT) approach to Random Forests (RF) --- an arisen meta-ensemble technique called Ant Colony Decision Forest (ACDF). To the best of our knowledge this is the first time that Ant Colony Optimization is being applied as an ensemble method in data mining tasks. Meta-ensemble ACDF as a hybrid RF and ACO based algorithm is evolved and experimentally shown high accuracy and good effectiveness of this technique motivate us to further development.