Javier G. Castellano

Bayesian network learning algorithms using structural restrictions

The use of several types of structural restrictions within algorithms for learning Bayesian networks is considered. These restrictions may codify expert knowledge in a given domain, in such a way that a Bayesian network representing this domain should satisfy them. The main goal of this paper is to study whether the algorithms for automatically learning the structure of a Bayesian network from data can obtain better results by using this prior knowledge. Three types of restrictions are formally defined: existence of arcs and/or edges, absence of arcs and/or edges, and ordering restrictions. We analyze the possible interactions between these types of restrictions and also how the restrictions can be managed within Bayesian network learning algorithms based on both the score+search and conditional independence paradigms. Then we particularize our study to two classical learning algorithms: a local search algorithm guided by a scoring function, with the operators of arc addition, arc removal and arc reversal, and the PC algorithm. We also carry out experiments using these two algorithms on several data sets.

Learning Bayesian Network Classifiers: Searching in a Space of Partially Directed Acyclic Graphs

There is a commonly held opinion that the algorithms for learning unrestricted types of Bayesian networks, especially those based on the score+search paradigm, are not suitable for building competitive Bayesian network-based classifiers. Several specialized algorithms that carry out the search into different types of directed acyclic graph (DAG) topologies have since been developed, most of these being extensions (using augmenting arcs) or modifications of the Naive Bayes basic topology. In this paper, we present a new algorithm to induce classifiers based on Bayesian networks which obtains excellent results even when standard scoring functions are used. The method performs a simple local search in a space unlike unrestricted or augmented DAGs. Our search space consists of a type of partially directed acyclic graph (PDAG) which combines two concepts of DAG equivalence: classification equivalence and independence equivalence. The results of exhaustive experimentation indicate that the proposed method can compete with state-of-the-art algorithms for classification.

A comparative study on base classifiers in ensemble methods for credit scoring

Abstract In the last years, the application of artificial intelligence methods on credit risk assessment has meant an improvement over classic methods. Small improvements in the systems about credit scoring and bankruptcy prediction can suppose great profits. Then, any improvement represents a high interest to banks and financial institutions. Recent works show that ensembles of classifiers achieve the better results for this kind of tasks. In this paper, it is extended a previous work about the selection of the best base classifier used in ensembles on credit data sets. It is shown that a very simple base classifier, based on imprecise probabilities and uncertainty measures, attains a better trade-off among some aspects of interest for this type of studies such as accuracy and area under ROC curve (AUC). The AUC measure can be considered as a more appropriate measure in this grounds, where the different type of errors have different costs or consequences. The results shown here present to this simple classifier as an interesting choice to be used as base classifier in ensembles for credit scoring and bankruptcy prediction, proving that not only the individual performance of a classifier is the key point to be selected for an ensemble scheme.

Bayesian networks classifiers for gene-expression data

In this work, we study the application of Bayesian networks classifiers for gene expression data in three ways: first, we made an exhaustive state-of-art of Bayesian classifiers and Bayesian classifiers induced from microarray data. Second, we propose a preprocessing scheme for gene expression data, to induce Bayesian classifiers. Third, we evaluate different Bayesian classifiers for this kind of data, including the C-RPDAG classifier presented by the authors.

Optimisation of Multilayer Perceptrons Using a Distributed Evolutionary Algorithm with SOAP

SOAP (simple object access protocol) is a protocol that allows the access to remote objects independently of the computer architecture and the language. A client using SOAP can send or receive objects, or access remote object methods. Unlike other remote procedure call methods, like XML-RPC or RMI, SOAP can use many different transport types (for instance, it could be called as a CGI or as sockets). In this paper an approach to evolutionary distributed optimisation of multilayer perceptrons (MLP) using SOAP and language Perl has been done.Obtained results show that the parallel version of the developed programs obtains similar or better results using much less time than the sequential version, obtaining a good speedup. Also it can be shown that obtained results are better than those obtained by other authors using different methods.

Genetic Algorithm Visualization Using Self-organizing Maps

This paper gives an overview of evolutionary computation visualization and describes the application of visualization to some well known multidimensional problems. Self-Organizing Maps (SOM) are used for multidimensional scaling and projection. We show how different ways of training the SOM make it more or less adequate for the visualization task.

Methods to determine the branching attribute in bayesian multinets classifiers

Bayesian multinets are a Bayesian networks extension where context-specific conditional independences can be represented. The main aim of this work is to study different methods to choose the distinguished attribute in Bayesian multinets when we use them in supervised classification tasks. We have used different approaches: a wrapper method and several filter methods. This will allow us to determine the most appropriate approach that meets our requirements of accuracy and/or time.

Improving the Naive Bayes Classifier via a Quick Variable Selection Method Using Maximum of Entropy

Variable selection methods play an important role in the field of attribute mining. The Naive Bayes (NB) classifier is a very simple and popular classification method that yields good results in a short processing time. Hence, it is a very appropriate classifier for very large datasets. The method has a high dependence on the relationships between the variables. The Info-Gain (IG) measure, which is based on general entropy, can be used as a quick variable selection method. This measure ranks the importance of the attribute variables on a variable under study via the information obtained from a dataset. The main drawback is that it is always non-negative and it requires setting the information threshold to select the set of most important variables for each dataset. We introduce here a new quick variable selection method that generalizes the method based on the Info-Gain measure. It uses imprecise probabilities and the maximum entropy measure to select the most informative variables without setting a threshold. This new variable selection method, combined with the Naive Bayes classifier, improves the original method and provides a valuable tool for handling datasets with a very large number of features and a huge amount of data, where more complex methods are not computationally feasible.

A Random Forest approach using imprecise probabilities

Abstract The Random Forest classifier has been considered as an important reference in the data mining area. The building procedure of its base classifier (a decision tree) is principally based on a randomization process of data and features; and on a split criterion, which uses classic precise probabilities, to quantify the gain of information. One drawback found on this classifier is that it has a bad performance when it is applied on data sets with class noise. Very recently, it is proved that a new criterion which uses imprecise probabilities and general uncertainty measures, can improve the performance of the classic split criteria. In this work, the base classifier of the Random Forest is modified using that new criterion, producing also a new single decision tree model. This model join with the randomization process of features is the base classifier of a new procedure similar to the Random Forest, called Credal Random Forest. The principal differences between those two models are presented. In an experimental study, it is shown that the new method represents an improvement of the Random Forest when both are applied on data sets without class noise. But this improvement is notably greater when they are applied on data sets with class noise.

Evolutionary Computation Visualization: Application to G-PROP

Software visualization is an area of computer science devoted to supporting the understanding and effective use of algorithms. The application of software visualization to Evolutionary Computation has been receiving increasing attention during the last few years. In this paper we apply visualization technique to an evolutionary algorithm for multilayer perceptron training. Our goal is to better understand its internal behavior in order to improve the evolutionary part of the method. As a result of applying this this technique several deficiencies in the method have been discovered.