Edwin Lughofer

Evolving Fuzzy Systems - Methodologies, Advanced Concepts and Applications

I. Introduction.- Part I - Basic Methodologies.- II. Basic Algorithms for EFS.- III. EFS Approaches for Regression and Classification.- Part II - Advanced Concepts.- IV. Towards Robust and Process-Save EFS.- V. On Improving Performance and Increasing Useability of EFS.- VI. Interpretability Issues in EFS.- Part III - Applications.- VII. Online System Identification and Prediction.- VIII. On-Line Fault and Anomaly Detection.- IX. Visual Inspection Systems.- X. Further (Potential) Application Fields.- Epilog - Achievements, Open Problems and New Challenges in EFS.

Evolving fuzzy classifiers using different model architectures

In this paper we present two novel approaches for on-line evolving fuzzy classifiers, called eClass and FLEXFIS-Class. Both methods can be applied with different model architectures, including single model (SM) with class labels as consequents, classification hyper-planes as consequents, and multi-model (MM) architecture. Additionally, eClass can have a multi-input-multi-output (MIMO) architecture with multiple hyper-planes as consequents of each fuzzy rule. The difference between MM and MIMO architectures is that the former one applies one separate and independent fuzzy rule-based (FRB) classifier for each class and is using an indicator labelling scheme, while the latter one applies a single FRB where the rules are MIMO rather than MISO. Both, eClass and FLEXFIS-Class methods are designed to work on a per-sample basis and are thus one-pass, incremental. Additionally, their structure (FRB) is evolving rather than fixed. It adapts their parameters in antecedent and consequent parts with any newly loaded sample. A special emphasis is placed on advanced issues for improving accuracy and robustness, including a thorough comparison between global and local learning of consequent functions, a novel approach for detecting of and reacting on drifts in the data streams and an enhanced outlier treatment strategy. The methods and their extensions according to the advanced issues are evaluated on one benchmark problem of handwritten images recognition as well as on a real-life problem of image classification framework, where images should be classified into good and bad ones during an on-line and interactive production process.

PANFIS: A Novel Incremental Learning Machine

Most of the dynamics in real-world systems are compiled by shifts and drifts, which are uneasy to be overcome by omnipresent neuro-fuzzy systems. Nonetheless, learning in nonstationary environment entails a system owning high degree of flexibility capable of assembling its rule base autonomously according to the degree of nonlinearity contained in the system. In practice, the rule growing and pruning are carried out merely benefiting from a small snapshot of the complete training data to truncate the computational load and memory demand to the low level. An exposure of a novel algorithm, namely parsimonious network based on fuzzy inference system (PANFIS), is to this end presented herein. PANFIS can commence its learning process from scratch with an empty rule base. The fuzzy rules can be stitched up and expelled by virtue of statistical contributions of the fuzzy rules and injected datum afterward. Identical fuzzy sets may be alluded and blended to be one fuzzy set as a pursuit of a transparent rule base escalating humans interpretability. The learning and modeling performances of the proposed PANFIS are numerically validated using several benchmark problems from real-world or synthetic datasets. The validation includes comparisons with state-of-the-art evolving neuro-fuzzy methods and showcases that our new method can compete and in some cases even outperform these approaches in terms of predictive fidelity and model complexity.

Handling drifts and shifts in on-line data streams with evolving fuzzy systems

In this paper, we present new approaches to handling drift and shift in on-line data streams with the help of evolving fuzzy systems (EFS), which are characterized by the fact that their structure (rule base and parameters) is not fixed and not pre-determined, but is extracted from data streams on-line and in an incremental manner. When dealing with so-called drifts and s hifts in data streams, one needs to take into account (1) automatic detection of drifts and shifts and (2) automatic reaction to the drifts and shifts. This is important to avoid interruptions in the learning process and downtrends in predictive accuracy. To address the first problem, we propose an approach based on the concept fuzzy rule age. The second problem is addressed by including gradual forgetting of (1) antecedent parts and (2) consequent parameters. The latter can be achieved by including a forgetting factor in the recursive local learning process of the parameters, whose value is automatically extracted based on the intensity of the shift/drift. For addressing the former problem, we introduce two alternative methods: one is based on the evolving density-based clustering (eClustering) used to form the antecedents in the eTS approach; the other is based on the automatic adaptation of the learning rate of the evolving vector quantization (eVQ) method used to form the antecedent in the FLEXFIS approach. The paper concludes with an empirical evaluation of the impact of the proposed approaches in (on-line) real-world data sets in which drifts and shifts occur.

On-line assurance of interpretability criteria in evolving fuzzy systems – Achievements, new concepts and open issues

Abstract In this position paper, we are discussing achievements and open issues in the interpretability of evolving fuzzy systems (EFS). In addition to pure on-line complexity reduction approaches, which can be an important direction for increasing the transparency of the evolved fuzzy systems, we examine the state-of-the-art and provide further investigations and concepts regarding the following interpretability aspects: distinguishability, simplicity, consistency, coverage and completeness, feature importance levels, rule importance levels and interpretation of consequents. These are well-known and widely accepted criteria for the interpretability of expert-based and standard data-driven fuzzy systems in batch mode. So far, most have been investigated only rudimentarily in the context of evolving fuzzy systems, trained incrementally from data streams: EFS have focussed mainly on precise modeling, aiming for models of high predictive quality. Only in a few cases, the integration of complexity reduction steps has been handled. This paper thus seeks to close this gap by pointing out new ways of making EFS more transparent and interpretable within the scope of the criteria mentioned above. The role of knowledge expansion, a peculiar concept in EFS, will be also addressed. One key requirement in our investigations is the availability of all concepts for on-line usage, which means they should be incremental or at least allow fast processing.

Generalized smart evolving fuzzy systems

AbstractIn this paper, we propose a new methodology for learning evolving fuzzy systems (EFS) from data streams in terms of on-line regression/system identification problems. It comes with enhanced dynamic complexity reduction steps, acting on model components and on the input structure and by employing generalized fuzzy rules in arbitrarily rotated position. It is thus termed as Gen-Smart-EFS (GS-EFS), short for generalized smart evolving fuzzy systems. Equipped with a new projection concept for high-dimensional kernels onto one-dimensional fuzzy sets, our approach is able to provide equivalent conventional TS fuzzy systems with axis-parallel rules, thus maintaining interpretability when inferring new query samples. The on-line complexity reduction on rule level integrates a new merging concept based on a combined adjacency–homogeneity relation between two clusters (rules). On input structure level, complexity reduction is motivated by a combined statistical-geometric concept and acts in a smooth and soft manner by incrementally adapting feature weights: features may get smoothly out-weighted over time (

GENEFIS: Toward an Effective Localist Network

\rightarrow

Autonomous data stream clustering implementing split-and-merge concepts - Towards a plug-and-play approach

→soft on-line dimension reduction) but also may become reactivated at a later stage. Out-weighted features will contribute little to the rule evolution criterion, which prevents the generation of unnecessary rules and reduces over-fitting due to curse of dimensionality. The criterion relies on a newly developed re-scaled Mahalanobis distance measure for assuring monotonicity between feature weights and distance values. Gen-Smart-EFS will be evaluated based on high-dimensional real-world data (streaming) sets and compared with other well-known (evolving) fuzzy systems approaches. The results show improved accuracy with lower rule base complexity as well as smaller rule length when using Gen-Smart-EFS.