Pietro Ducange

A Multiobjective Evolutionary Approach to Concurrently Learn Rule and Data Bases of Linguistic Fuzzy-Rule-Based Systems

In this paper, we propose the use of a multiobjective evolutionary approach to generate a set of linguistic fuzzy-rule-based systems with different tradeoffs between accuracy and interpretability in regression problems. Accuracy and interpretability are measured in terms of approximation error and rule base (RB) complexity, respectively. The proposed approach is based on concurrently learning RBs and parameters of the membership functions of the associated linguistic labels. To manage the size of the search space, we have integrated the linguistic two-tuple representation model, which allows the symbolic translation of a label by only considering one parameter, with an efficient modification of the well known (2 + 2) Pareto archived evolution strategy (PAES). We tested our approach on nine real world datasets of different sizes and with different numbers of variables. Besides the (2 + 2)PAES, we have also used the well known nondominated sorting genetic algorithm (NSGA-II) and an accuracy-driven single-objective evolutionary algorithm (EA). We employed these optimization techniques both to concurrently learn rules and parameters and to learn only rules. We compared the different approaches by applying a nonparametric statistical test for pairwise comparisons, thus taking into consideration three representative points from the obtained Pareto fronts in the case of the multiobjective EAs. Finally, a data complexity measure, which is typically used in pattern recognition to evaluate the data density in terms of average number of patterns per variable, has been introduced to characterize regression problems. Results confirm the effectiveness of our approach, particularly for (possibly high dimensional) datasets with high values of the complexity metric.

A Pareto-based multi-objective evolutionary approach to the identification of Mamdani fuzzy systems

In the last years, the numerous successful applications of fuzzy rule-based systems (FRBSs) to several different domains have produced a considerable interest in methods to generate FRBSs from data. Most of the methods proposed in the literature, however, focus on performance maximization and omit to consider FRBS comprehensibility. Only recently, the problem of finding the right trade-off between performance and comprehensibility, in spite of the original nature of fuzzy logic, has arisen a growing interest in methods which take both the aspects into account. In this paper, we propose a Pareto-based multi-objective evolutionary approach to generate a set of Mamdani fuzzy systems from numerical data. We adopt a variant of the well-known (2+2) Pareto Archived Evolutionary Strategy ((2+2)PAES), which adopts the one-point crossover and two appropriately defined mutation operators. (2+2)PAES determines an approximation of the optimal Pareto front by concurrently minimizing the root mean squared error and the complexity. Complexity is measured as sum of the conditions which compose the antecedents of the rules included in the FRBS. Thus, low values of complexity correspond to Mamdani fuzzy systems characterized by a low number of rules and a low number of input variables really used in each rule. This ensures a high comprehensibility of the systems. We tested our version of (2+2)PAES on three well-known regression benchmarks, namely the Box and Jenkins Gas Furnace, the Mackey-Glass chaotic time series and Lorenz attractor time series datasets. To show the good characteristics of our approach, we compare the Pareto fronts produced by the (2+2)PAES with the ones obtained by applying a heuristic approach based on SVD-QR decomposition and four different multi-objective evolutionary algorithms.

Multi-objective genetic fuzzy classifiers for imbalanced and cost-sensitive datasets

We exploit an evolutionary three-objective optimization algorithm to produce a Pareto front approximation composed of fuzzy rule-based classifiers (FRBCs) with different trade-offs between accuracy (expressed in terms of sensitivity and specificity) and complexity (computed as sum of the conditions in the antecedents of the classifier rules). Then, we use the ROC convex hull method to select the potentially optimal classifiers in the projection of the Pareto front approximation onto the ROC plane. Our method was tested on 13 highly imbalanced datasets and compared with 2 two-objective evolutionary approaches and one heuristic approach to FRBC generation, and with three well-known classifiers. We show by the Wilcoxon signed-rank test that our three-objective optimization approach outperforms all the other techniques, except for one classifier, in terms of the area under the ROC convex hull, an accuracy measure used to globally compare different classification approaches. Further, all the FRBCs in the ROC convex hull are characterized by a low value of complexity. Finally, we discuss how, the misclassification costs and the class distributions are fixed, we can select the most suitable classifier for the specific application. We show that the FRBC selected from the convex hull produced by our three-objective optimization approach achieves the lowest classification cost among the techniques used as comparison in two specific medical applications.

Real-Time Detection of Traffic From Twitter Stream Analysis

Social networks have been recently employed as a source of information for event detection, with particular reference to road traffic congestion and car accidents. In this paper, we present a real-time monitoring system for traffic event detection from Twitter stream analysis. The system fetches tweets from Twitter according to several search criteria; processes tweets, by applying text mining techniques; and finally performs the classification of tweets. The aim is to assign the appropriate class label to each tweet, as related to a traffic event or not. The traffic detection system was employed for real-time monitoring of several areas of the Italian road network, allowing for detection of traffic events almost in real time, often before online traffic news web sites. We employed the support vector machine as a classification model, and we achieved an accuracy value of 95.75% by solving a binary classification problem (traffic versus nontraffic tweets). We were also able to discriminate if traffic is caused by an external event or not, by solving a multiclass classification problem and obtaining an accuracy value of 88.89%.

Learning concurrently partition granularities and rule bases of Mamdani fuzzy systems in a multi-objective evolutionary framework

In this paper we propose a multi-objective evolutionary algorithm to generate Mamdani fuzzy rule-based systems with different good trade-offs between complexity and accuracy. The main novelty of the algorithm is that both rule base and granularity of the uniform partitions defined on the input and output variables are learned concurrently. To this aim, we introduce the concepts of virtual and concrete rule bases: the former is defined on linguistic variables, all partitioned with a fixed maximum number of fuzzy sets, while the latter takes into account, for each variable, a number of fuzzy sets as determined by the specific partition granularity of that variable. We exploit a chromosome composed of two parts, which codify the variables partition granularities, and the virtual rule base, respectively. Genetic operators manage virtual rule bases, whereas fitness evaluation relies on an appropriate mapping strategy between virtual and concrete rule bases. The algorithm has been tested on two real-world regression problems showing very promising results.

Genetic Training Instance Selection in Multiobjective Evolutionary Fuzzy Systems: A Coevolutionary Approach

When dealing with datasets that are characterized by a large number of instances, multiobjective evolutionary learning (MOEL) of fuzzy rule-based systems (FRBSs) suffers from high computational costs, mainly because of the fitness evaluation. The use of a reduced set of representative instances in place of the overall training set (TS) would considerably lessen the computational effort. Even though a large number of papers have proposed instance selection approaches, mainly in classification problems, how this selection should be performed, especially in the context of regression, is still an open issue. In this paper, we tackle the instance selection problem in the framework of MOEL of FRBSs through a coevolutionary approach. In the execution of the MOEL, periodically, a single-objective genetic algorithm (SOGA) evolves a population of reduced TSs. The SOGA aims to maximize a purposely defined index which measures how much the Pareto fronts computed by using, respectively, the reduced TS and the overall TS are close to each other: The closer the fronts, the more the reduced TS is representative of the overall TS. During the execution of the MOEL, the rule base and the membership function parameters of the fuzzy sets are concurrently learned by maximizing the accuracy and minimizing the complexity. We tested our approach on 12 large datasets. We adopted reduced TSs composed of 5%, 10%, and 20% of the overall TS. Using nonparametric statistical tests, we verified that with 10% and 20% of the overall TS, the Pareto front approximations that are generated by our coevolutionary approach are comparable with the ones generated by applying the MOEL with the overall TS, although the coevolution allows us to save up to 86.36% of the execution time. In addition, the analysis of the behavior of three representative solutions on the test set highlights that the use of the reduced TSs does not affect the generalization capabilities of the generated FRBSs.

Learning knowledge bases of multi-objective evolutionary fuzzy systems by simultaneously optimizing accuracy, complexity and partition integrity

In the last few years, several papers have exploited multi-objective evolutionary algorithms (MOEAs) to generate Mamdani fuzzy rule-based systems (MFRBSs) with different trade-offs between interpretability and accuracy. In this framework, a common approach is to distinguish between interpretability of the rule base (RB), also known as complexity, and interpretability of fuzzy partitions, also known as integrity of the database (DB). Typically, complexity has been used as one of the objectives of the MOEAs, while partition integrity has been ensured by enforcing constraints on the membership function (MF) parameters. In this paper, we propose to adopt partition integrity as an objective of the evolutionary process. To this aim, we first discuss how partition integrity can be measured by using a purposely defined index based on the similarity between the partitions learned during the evolutionary process and the initial interpretable partitions defined by an expert. Then, we introduce a three-objective evolutionary algorithm which generates a set of MFRBSs with different trade-offs between complexity, accuracy and partition integrity by concurrently learning the RB and the MF parameters of the linguistic variables. Accuracy is assessed in terms of mean squared error between the actual and the predicted values, complexity is calculated as the total number of conditions in the antecedents of the rules and integrity is measured by using the purposely defined index. The proposed approach has been experimented on six real-world regression problems. The results have been compared with those obtained by applying the same MOEA, but with only accuracy and complexity as objectives, both to learn only RBs, and to concurrently learn RBs and MF parameters, with and without constraints on the parameter tuning. We show that our approach achieves the best trade-offs between interpretability and accuracy. Finally, we compare our approach with a similar MOEA recently proposed in the literature.

An intelligent system for detecting faults in photovoltaic fields

In this work, an intelligent system for automatic detection of fault in PV fields is proposed. This system is based on a Takagi-Sugeno-Kahn Fuzzy Rule-Based System (TSK-FRBS), which provides an estimation of the instant power production of the PV field in normal functioning, i.e, when no faults occur. Then, the estimated power is compared with the real power and an alarm signal is generated if the difference between powers overcomes a threshold. The TSK-FRBS has been trained using data collected from a PV plant simulator, during normal functioning. Preliminary tests were carried out in a simulated framework, by reproducing both normal and fault conditions. Results show that the system can recognize more than 90% of fault conditions, even when noisy data are introduced.

Multi-objective evolutionary learning of granularity, membership function parameters and rules of Mamdani fuzzy systems

In this paper, we propose a multi-objective evolutionary algorithm (MOEA) to generate Mamdani fuzzy rule-based systems with different trade-offs between accuracy and complexity by learning concurrently granularities of the input and output partitions, membership function (MF) parameters and rules. To this aim, we introduce the concept of virtual and concrete partitions: the former is defined by uniformly partitioning each linguistic variable with a fixed maximum number of fuzzy sets; the latter takes into account, for each variable, the number of fuzzy sets determined by the evolutionary process. Rule bases and MF parameters are defined on the virtual partitions and, whenever a fitness evaluation is required, mapped to the concrete partitions by employing appropriate mapping strategies. The implementation of the MOEA relies on a chromosome composed of three parts, which codify the partition granularities, the virtual rule base and the membership function parameters, respectively, and on purposely-defined genetic operators. The MOEA has been tested on three real-world regression problems achieving very promising results. In particular, we highlight how starting from randomly generated solutions, the MOEA is able to determine different granularities for different variables achieving good trade-offs between complexity and accuracy.

A fast and efficient multi-objective evolutionary learning scheme for fuzzy rule-based classifiers

During the last years, multi-objective evolutionary algorithms (MOEAs) have been extensively used to generate fuzzy rule-based systems characterized by different trade-offs between accuracy and complexity. In this paper, we propose an MOEA-based approach to learn concurrently the rule and data bases of fuzzy rule-based classifiers (FRBCs). In particular, the rule bases are generated by exploiting a rule and condition selection (RCS) strategy, which selects a reduced number of rules from a heuristically generated set of candidate rules and a reduced number of conditions for each selected rule during the evolutionary process. RCS can be considered as a rule learning in a constrained search space. As regards the data base learning, the membership function parameters of each linguistic term used in the rules are learned concurrently to the application of RCS. We tested our approach on twenty-four classification benchmarks and compared our results with the ones obtained by two similar state-of-the-art MOEA-based approaches and by two well-known non-evolutionary classification algorithms, namely FURIA and C4.5. Using non-parametric statistical tests, we show that our approach generates FRBCs with accuracy and complexity statistically comparable to, and sometimes better than, the ones generated by the two MOEA-based approaches, exploiting, however, only the 5% of the number of fitness evaluations used by these approaches. Further, the classifiers generated by our approach result to be more interpretable than the ones generated by the FURIA and C4.5 algorithms, while achieving the same accuracy level.