Online bagging of evolving fuzzy systems

Abstract

Abstract Evolving fuzzy systems (EFS) have received increased attention from the community for the purpose of data stream modeling in an incremental, single-pass and transparent manner. To date, a wide variety of EFS approaches have been developed and successfully used in real-world applications which address structural evolution and parameter adaptation in single EFS models. We propose a specific ensemble scheme of EFS to increase their robustness in predictive performance on new stream samples. Our approach relies on an online variant of bagging in which various EFS ensemble members are generated from online bags, that is, the members are updated based on a specific probabilistic online sampling technique, and this with guaranteed convergence to classical sampling in batch bagging. The autonomous pruning of ensemble members is undertaken to omit undesired members with atypically higher errors than other members. We propose two variants, hard pruning where undesired members are deleted forever from the ensemble, and soft pruning where members receive weights to calculate the overall ensemble prediction, according to their single performance; thus, members who are undesired at a certain point of time may be dynamically recalled at a later stage. The autonomous evolution of new ensemble members is carried out whenever a drift in the stream is detected, based on a significantly worsening performance indicator, measured in terms of the Hoeffding inequality. Newer members typically represent the drifted state better and are thus up-weighed compared to older members within an advanced (weighted) calculation of the overall ensemble prediction. The new approach termed online bagged EFS (OB-EFS) was successfully evaluated and compared with single EFS models and related SoA approaches on four data streams from real-world applications (containing various noise levels, drifts and new operating conditions) and showed significantly lower prediction error trend lines.