Luciano Sánchez

Combining GP operators with SA search to evolve fuzzy rule based classifiers

Abstract The genotype–phenotype encoding of fuzzy rule bases in GA, along with their corresponding crossover and mutation operators, can be used by other search schemes, improving the behavior of these last ones. As a practical consequence of this, a simulated annealing-based method for inducting both parameters and structure of a fuzzy classifier has been developed. The adjacency operator in SA has been replaced with a macromutation taken from tree-shaped genotype GAs. We will show that results of SA search are similar to those of GP in both the efficiency of the learned classifiers and in its linguistic interpretability, while the memory consumption of the learning process is lower.

Induction of fuzzy-rule-based classifiers with evolutionary boosting algorithms

This paper proposes a novel Adaboost algorithm to learn fuzzy-rule-based classifiers. Connections between iterative learning and boosting are analyzed in terms of their respective structures and the manner these algorithms address the cooperation-competition problem. The results are used to explain some properties of the former method. The evolutionary boosting scheme is applied to approximate and descriptive fuzzy-rule bases. The advantages of boosting fuzzy rules are assessed by performance comparisons between the proposed method and other classification schemes applied on a set of benchmark classification tasks.

Solving Electrical Distribution Problems Using Hybrid Evolutionary Data Analysis Techniques

Real-world electrical engineering problems can take advantage of the last Data Analysis methodologies. In this paper we will show that Genetic Fuzzy Rule-Based Systems and Genetic Programming techniques are good choices for tackling with some practical modeling problems. We claim that both evolutionary processes may produce good numerical results while providing us with a model that can be interpreted by a human being. We will analyze in detail the characteristics of these two methods and we will compare them to the some of the most popular classical statistical modeling methods and neural networks.

Genetic learning of fuzzy rules based on low quality data

Genetic fuzzy systems (GFS) are based on the use of genetic algorithms for designing fuzzy systems, and for providing them with learning and adaptation capabilities. In this context, fuzzy sets represent linguistic granules of information, contained in the antecedents and consequents of the rules, whereas the data used in the genetic learning is assumed to be crisp. GFS seldom deal with fuzzy-valued data. In this paper we address this problem, and propose a set of techniques that can be incorporated to different GFS in order to learn a knowledge base (KB) from interval and fuzzy data for regression problems. Details will be given about the representation of non-standard data with fuzzy sets, about the needed changes in the reasoning method of the fuzzy rule-based system, and also about a new generalization of the mean squared error to vague data. In addition, we will show that the learning process requires a genetic algorithm that must be capable of optimizing a multicriteria fitness function, containing both crisp and interval-valued criteria. Lastly, we benchmark our procedures with some machine learning related datasets and a real-world problem of marketing, and the techniques proposed here are shown to improve the generalization properties of other KBs obtained from crisp training data.

Advocating the Use of Imprecisely Observed Data in Genetic Fuzzy Systems

In our opinion, and in accordance with current literature, the precise contribution of genetic fuzzy systems to the corpus of the machine learning theory has not been clearly stated yet. In particular, we question the existence of a set of problems for which the use of fuzzy rules, in combination with genetic algorithms, produces more robust models, or classifiers that are inherently better than those arising from the Bayesian point of view. We will show that this set of problems actually exists, and comprises interval and fuzzy valued datasets, but it is not being exploited. Current genetic fuzzy classifiers deal with crisp classification problems, where the role of fuzzy sets is reduced to give a parametric definition of a set of discriminant functions, with a convenient linguistic interpretation. Provided that the customary use of fuzzy sets in statistics is vague data, we propose to test genetic fuzzy classifiers over imprecisely measured data and design experiments well suited to these problems. The same can be said about genetic fuzzy models: the use of a scalar fitness function assumes crisp data, where fuzzy models, a priori, do not have advantages over statistical regression.

Higher order models for fuzzy random variables

A fuzzy random variable is viewed as the imprecise observation of the outcomes in a random experiment. Since randomness and vagueness coexist in the same framework, it seems reasonable to integrate fuzzy random variables into imprecise probabilities theory. Nevertheless, fuzzy random variables are commonly presented in the literature as classical measurable functions associated to a classical probability measure. We present here a higher order possibility model that represents the imprecise information provided by a fuzzy random variable. We compare it with previous classical models in the literature. First, some aspects about the acceptability function associated to a fuzzy random variable are investigated. Secondly, we present three different higher order possibility models, all of them arising in a natural way. We investigate their similarities and differences, and observe that the first one (the fuzzy probability envelope) is the most informative. Finally we compare the fuzzy probability envelope with the (classical) probability measure induced by the fuzzy random variable. We conclude that the classical probability measure does not always contain all relevant information provided by a fuzzy random variable.

Mutual information-based feature selection and partition design in fuzzy rule-based classifiers from vague data

Algorithms for preprocessing databases with incomplete and imprecise data are seldom studied. For the most part, we lack numerical tools to quantify the mutual information between fuzzy random variables. Therefore, these algorithms (discretization, instance selection, feature selection, etc.) have to use crisp estimations of the interdependency between continuous variables, whose application to vague datasets is arguable. In particular, when we select features for being used in fuzzy rule-based classifiers, we often use a mutual information-based ranking of the relevance of inputs. But, either with crisp or fuzzy data, fuzzy rule-based systems route the input through a fuzzification interface. The fuzzification process may alter this ranking, as the partition of the input data does not need to be optimal. In our opinion, to discover the most important variables for a fuzzy rule-based system, we want to compute the mutual information between the fuzzified variables, and we should not assume that the ranking between the crisp variables is the best one. In this paper we address these problems, and propose an extended definition of the mutual information between two fuzzified continuous variables. We also introduce a numerical algorithm for estimating the mutual information from a sample of vague data. We will show that this estimation can be included in a feature selection algorithm, and also that, in combination with a genetic optimization, the same definition can be used to obtain the most informative fuzzy partition for the data. Both applications will be exemplified with the help of some benchmark problems.

Diagnosis of dyslexia with low quality data with genetic fuzzy systems

For diagnosing dyslexia in early childhood, children have to solve non-writing based graphical tests. Human experts score these tests, and decide whether the children require further consideration on the basis of their marks. Applying artificial intelligence techniques for automating this scoring is a complex task with multiple sources of uncertainty. On the one hand, there are conflicts, as different experts can assign different scores to the same set of answers. On the other hand, sometimes the experts are not completely confident with their decisions and doubt between different scores. The problem is aggravated because certain symptoms are compatible with more than one disorder. In case of doubt, the experts assign an interval-valued score to the test and ask for further information about the child before diagnosing him. Having said that, exploiting the information in uncertain datasets has been recently acknowledged as a new challenge in genetic fuzzy systems. In this paper we propose using a genetic cooperative-competitive algorithm for designing a linguistically understandable, rule-based classifier that can tackle this problem. This algorithm is part of a web-based, automated pre-screening application that can be used by the parents for detecting those symptoms that advise taking the children to a psychologist for an individual examination.

Obtaining linguistic fuzzy rule-based regression models from imprecise data with multiobjective genetic algorithms

Backfitting of fuzzy rules is an Iterative Rule Learning technique for obtaining the knowledge base of a fuzzy rule-based system in regression problems. It consists in fitting one fuzzy rule to the data, and replacing the whole training set by the residual of the approximation. The obtained rule is added to the knowledge base, and the process is repeated until the residual is zero, or near zero. Such a design has been extended to imprecise data for which the observation error is small. Nevertheless, when this error is moderate or high, the learning can stop early. In this kind of algorithms, the specificity of the residual might decrease when a new rule is added. There may happen that the residual grows so wide that it covers the value zero for all points (thus the algorithm stops), but we have not yet extracted all the information available in the dataset. Focusing on this problem, this paper is about datasets with medium to high discrepancies between the observed and the actual values of the variables, such as those containing missing values and coarsely discretized data. We will show that the quality of the iterative learning degrades in this kind of problems, because it does not make full use of all the available information. As an alternative to sequentially obtaining rules, we propose a new multiobjective Genetic Cooperative Competitive Learning (GCCL) algorithm. In our approach, each individual in the population codifies one rule, which competes in the population in terms of maximum coverage and fitting, while the individuals in the population cooperate to form the knowledge base.

Interval-valued GA-P algorithms

When genetic programming (GP) methods are applied to solve symbolic regression problems, we obtain a point estimate of a variable, but it is not easy to calculate an associated confidence interval. We designed an interval arithmetic-based model that solves this problem. Our model extends a hybrid technique, the GA-P method, that combines genetic algorithms and genetic programming. Models based on interval GA-P can devise an interval model from examples and provide the algebraic expression that best approximates the data. The method is useful for generating a confidence interval for the output of a model, and also for obtaining a robust point estimate from data which we know to contain outliers. The algorithm was applied to a real problem related to electrical energy distribution. Classical methods were applied first, and then the interval GA-P. The results of both studies are used to compare interval GA-P with GP, GA-P, classical regression methods, neural networks, and fuzzy models.