Magne Setnes

GA-fuzzy modeling and classification: complexity and performance

The use of genetic algorithms (GAs) and other evolutionary optimization methods to design fuzzy rules for systems modeling and data classification have received much attention in recent literature. Authors have focused on various aspects of these randomized techniques, and a whole scale of algorithms have been proposed. We comment on some recent work and describe a new and efficient two-step approach that leads to good results for function approximation, dynamic systems modeling and data classification problems. First, fuzzy clustering is applied to obtain a compact initial rule-based model. Then this model is optimized by a real-coded GA subjected to constraints that maintain the semantic properties of the rules. We consider four examples from the literature: a synthetic nonlinear dynamic systems model, the iris data classification problem, the wine data classification problem, and the dynamic modeling of a diesel engine turbocharger. The obtained results are compared to other recently proposed methods.

Similarity measures in fuzzy rule base simplification

In fuzzy rule-based models acquired from numerical data, redundancy may be present in the form of similar fuzzy sets that represent compatible concepts. This results in an unnecessarily complex and less transparent linguistic description of the system. By using a measure of similarity, a rule base simplification method is proposed that reduces the number of fuzzy sets in the model. Similar fuzzy sets are merged to create a common fuzzy set to replace them in the rule base. If the redundancy in the model is high, merging similar fuzzy sets might result in equal rules that also can be merged, thereby reducing the number of rules as well. The simplified rule base is computationally more efficient and linguistically more tractable. The approach has been successfully applied to fuzzy models of real world systems.

Rule-based modeling: precision and transparency

This article is a reaction to recent publications on rule-based modeling using fuzzy set theory and fuzzy logic. The interest in fuzzy systems has recently shifted from the seminal ideas about complexity reduction toward data-driven construction of fuzzy systems. Many algorithms have been introduced that aim at numerical approximation of functions by rules, but pay little attention to the interpretability of the resulting rule base. We show that fuzzy rule-based models acquired from measurements can be both accurate and transparent by using a low number of rules. The rules are generated by product-space clustering and describe the system in terms of the characteristic local behavior of the system in regions identified by the clustering algorithm. The fuzzy transition between rules makes it possible to achieve precision along with a good qualitative description in linguistic terms. The latter is useful for expert evaluation, rule-base maintenance, operator training, control systems design, user interfacing, etc. We demonstrate the approach on a modeling problem from a recently published article.

Compact and transparent fuzzy models and classifiers through iterative complexity reduction

In our previous work (2000) we showed that genetic algorithms (GAs) provide a powerful tool to increase the accuracy of fuzzy models for both systems modeling and classification. In addition to these results, we explore the GA to find redundancy in the fuzzy model for the purpose of model reduction. An aggregated similarity measure is applied to search for redundancy in the rule base description. As a result, we propose an iterative fuzzy identification technique starting with data-based fuzzy clustering with an overestimated number of local models. The GA is then applied to find redundancy among the local models with a criterion based on maximal accuracy and maximal set similarity. After the reduction steps, the GA is applied with another criterion searching for minimal set similarity and maximal accuracy. This results in an automatic identification scheme with fuzzy clustering, rule base simplification and constrained genetic optimization with low-human intervention. The proposed modeling approach is then demonstrated for a system identification and a classification problem.

Supervised fuzzy clustering for rule extraction

The paper is concerned with the application of orthogonal transforms and fuzzy clustering to the extraction of fuzzy rules from data. It is proposed to use the orthogonal least squares method to supervise the progress of the fuzzy clustering algorithm and remove clusters of less importance with respect to fitting the data. Clustering takes place in the product space of systems in and outputs, and each cluster corresponds to a fuzzy IF-THEN rule. By initializing the clustering with an overestimated number of clusters, and subsequently remove less important clusters (rules) as the clustering progresses, it is sought to obtain a suitable partition of the data in an automated. The approach is studied for the fuzzy c-means algorithm and applied to a function approximation example known from the literature.

Fuzzy clustering with volume prototypes and adaptive cluster merging

Two extensions to objective function-based fuzzy clustering are proposed. First, the (point) prototypes are extended to hypervolumes, whose size can be fixed or can be determined automatically from the data being clustered. It is shown that clustering with hypervolume prototypes can be formulated as the minimization of an objective function. Second, a heuristic cluster merging step is introduced where the similarity among the clusters is assessed during optimization. Starting with an overestimation of the number of clusters in the data, similar clusters are merged in order to obtain a suitable partitioning. An adaptive threshold for merging is proposed. The extensions proposed are applied to Gustafson-Kessel and fuzzy c-means algorithms, and the resulting extended algorithm is given. The properties of the new algorithm are illustrated by various examples.

Learning Fuzzy Classification Rules from Data

Automatic design of fuzzy rule-based classification systems based on labeled data is considered. It is recognized that both classification performance and interpretability are of major importance and effort is made to keep the resulting rule bases small and comprehensible. An iterative approach for developing fuzzy classifiers is proposed. The initial model is derived from the data and subsequently, feature selection and rule base simplification are applied to reduce the model, and a GA is used for model tuning. An application to the Wine data classification problem is shown.

Fuzzy modeling of client preference from large data sets: an application to target selection in direct marketing

Advances in computational methods have led, in the world of financial services, to huge databases of client and market information. In the past decade, various computational intelligence techniques have been applied in mining this data for obtaining knowledge and in-depth information about the clients and the markets. The paper discusses the application of fuzzy clustering in target selection from large databases for direct marketing purposes. Actual data from the campaigns of a large financial services provider are used as a test case. The results obtained with the fuzzy clustering approach are compared with those resulting from the current practice of using statistical tools for target selection.

Transparent fuzzy modeling using fuzzy clustering and GAs

A combined approach to data-driven fuzzy rule-based modeling is described. The rules of an initial model are derived from data by means of a supervised clustering method that to a certain degree ensures the transparency of the resulting rule base. This model is, however suboptimal and a real-coded genetic algorithm (GA) is proposed to optimize simultaneously both the antecedent and the consequent variables. The GA is subjected to constraints concerning the semantic properties of the rule base, inherited from the initial model. Two modeling problems illustrate the power of the combined approach.

Transparent Fuzzy Modelling

One of the objectives of machine learning is to enable intelligent systems to acquire knowledge in a highly automated manner. In systems modelling and control engineering, fuzzy systems have shown to be highly suitable for the modelling of complex and uncertain systems. Recently, the interest in fuzzy systems has shifted from the seminal ideas about modelling the process or the behaviour of operators by knowledge acquisition towards a data-driven approach. Reasons to choose fuzzy systems instead of modelling techniques such as neural networks, radial basis functions, genetic algorithms or splines, are mainly the possibility of integrating logical information processing with the attractive mathematical properties of general function approximators. Furthermore, the rule-based structure of fuzzy systems makes analysis easier. The fuzzy sets in the rules represent linguistic qualitative terms that approximate the human-like way of information quantization. However, many of the data-driven fuzzy modelling algorithms that have been developed, aim at good numerical approximation and pay little attention to the semantical properties of the resulting rule base. In this article, we briefly discuss different approaches to data-intensive fuzzy modelling reported in the literature. Next, we present a data-driven approach to fuzzy modelling that provides the user with both accurate and transparent rule bases. The method has two main steps: data exploration by means of fuzzy clustering and fuzzy set aggregation by means of similarity analysis. First, fuzzy relations are identified in the product space of the systems variables and are described by means of fuzzy production rules. Compatible fuzzy concepts defined for the individual variables are then identified and aggregated to produce generalizing concepts, giving a comprehensible rule base with increased semantic properties. The transparent fuzzy modelling approach is demonstrated on a real world problem concerning the modelling of algae growth in lakes.