Giovanna Castellano

An iterative pruning algorithm for feedforward neural networks

The problem of determining the proper size of an artificial neural network is recognized to be crucial, especially for its practical implications in such important issues as learning and generalization. One popular approach for tackling this problem is commonly known as pruning and it consists of training a larger than necessary network and then removing unnecessary weights/nodes. In this paper, a new pruning method is developed, based on the idea of iteratively eliminating units and adjusting the remaining weights in such a way that the network performance does not worsen over the entire training set. The pruning problem is formulated in terms of solving a system of linear equations, and a very efficient conjugate gradient algorithm is used for solving it, in the least-squares sense. The algorithm also provides a simple criterion for choosing the units to be removed, which has proved to work well in practice. The results obtained over various test problems demonstrate the effectiveness of the proposed approach.

Variable selection using neural-network models

Abstract In this paper we propose an approach to variable selection that uses a neural-network model as the tool to determine which variables are to be discarded. The method performs a backward selection by successively removing input nodes in a network trained with the complete set of variables as inputs. Input nodes are removed, along with their connections, and remaining weights are adjusted in such a way that the overall input–output behavior learnt by the network is kept approximately unchanged. A simple criterion to select input nodes to be removed is developed. The proposed method is tested on a famous example of system identification. Experimental results show that the removal of input nodes from the neural network model improves its generalization ability. In addition, the method compares favorably with respect to other feature reduction methods.

Knowledge discovery by a neuro-fuzzy modeling framework

In this paper a neuro-fuzzy modeling framework is proposed, which is devoted to discover knowledge from data and represent it in the form of fuzzy rules. The core of the framework is a knowledge extraction procedure that is aimed to identify the structure and the parameters of a fuzzy rule base, through a two-phase learning of a neuro-fuzzy network. In order to obtain reliable and readable knowledge, two further stages are integrated with the knowledge extraction procedure: a pre-processing stage, performing variable selection on the available data to obtain simpler and more reliable fuzzy rules, and a post-processing stage, that granulates outputs of the extracted fuzzy rules so as to provide a validity range of estimated outputs. Moreover, the framework can address complex multi-input multi-output problems. In such case, two distinct modeling strategies can be followed with the opportunity of producing both a single MIMO model or a collection of MISO models. The proposed framework is verified on a real-world case study, involving prediction of chemical composition of ashes produced by combustion processes carried out in thermo-electric generators located in Italy.

Distinguishability quantification of fuzzy sets

Distinguishability is a semantic property of fuzzy sets that has a great relevance in the design of interpretable fuzzy models. Distinguishability has been mathematically defined through different measures, which are addressed in this paper. Special emphasis is given to similarity, which exhibits sound theoretical properties but its calculation is usually computationally intensive, and possibility, whose calculation can be very efficient but it does not exhibit the same properties of similarity. It is shown that under mild conditions – usually met in interpretable fuzzy modeling – possibility can be used as a valid measure for assessing distinguishability, thus overcoming the computational inefficiencies of similarity measures. Moreover, procedures that minimize possibility also minimize similarity and, consequently, improve distinguishability. In this sense, the use of possibility is fully justified in interpretable fuzzy modeling.

A neuro-fuzzy network to generate human-understandable knowledge from data

Neuro-fuzzy networks have been successfully applied to extract knowledge from data in the form of fuzzy rules. However, one drawback with the neuro-fuzzy approach is that the fuzzy rules induced by the learning process are not necessarily understandable. The lack of readability is essentially due to the high dimensionality of the parameter space that leads to excessive flexibility in the modification of parameters during learning. In this paper, to obtain readable knowledge from data, we propose a new neuro-fuzzy model and its learning algorithm that works in a parameter space with reduced dimensionality. The dimensionality of the new parameter space is necessary and sufficient to generate human-understandable fuzzy rules, in the sense formally defined by a set of properties. The learning procedure is based on a gradient descent technique and the proposed model is general enough to be applied to other neuro-fuzzy architectures. Simulation studies on a benchmark and a real-life problem are carried out to embody the idea of the paper.

Meta-data: characterization of input features for meta-learning

Common inductive learning strategies offer the tools for knowledge acquisition, but possess some inherent limitations due to the use of fixed bias during the learning process. To overcome limitations of such base-learning approaches, a novel research trend is oriented to explore the potentialities of meta-learning, which is oriented to the development of mechanisms based on a dynamical search of bias. This could lead to an improvement of the base-learner performance on specific learning tasks, by profiting of the accumulated past experience. As a significant set of I/O data is needed for efficient base-learning, appropriate meta-data characterization is of crucial importance for useful meta-learning. In order to characterize meta-data, firstly a collection of meta-features discriminating among different base-level tasks should be identified. This paper focuses on the characterization of meta-data, through an analysis of meta-features that can capture the properties of specific tasks to be solved at base level. This kind of approach represents a first step toward the development of a meta-learning system, capable of suggesting the proper bias for base-learning different specific task domains.

An empirical risk functional to improve learning in a neuro-fuzzy classifier

The paper proposes a new Empirical Risk Functional as cost function for training neuro-fuzzy classifiers. This cost function, called Approximate Differentiable Empirical Risk Functional (ADERF), provides a differentiable approximation of the misclassification rate so that the Empirical Risk Minimization Principle formulated in Vapniks Statistical Learning Theory can be applied. Also, based on the proposed ADERF, a learning algorithm is formulated. Experimental results on a number of benchmark classification tasks are provided and comparison to alternative approaches given.

Similarity-Based Fuzzy Clustering for User Profiling

User profiling is a fundamental task in Web personalization. Fuzzy clustering is a valid approach to derive user profiles by capturing similar user interests from Web usage data available in log files. Often, fuzzy clustering is based on the assumption that data lay on an Euclidean space; however, clustering based on Euclidean distance can lead the clustering process to find user representations that do not capture the semantic information incorporated in the original Web usage data. In this paper, we propose a different approach to express similarity between Web users. The measure is based on the evaluation of similarity between fuzzy sets. The proposed measure is employed in a relational fuzzy clustering algorithm to discover clusters embedded in the Web usage data and derive profiles modeling the real user preferences. An application example on usage data extracted from log files of a sample Web site is reported and a comparison with the results obtained using the cosine measure is shown to demonstrate the effectiveness of the proposed similarity measure.

NEWER: A system for NEuro-fuzzy WEb Recommendation

In the era of the Web, there is urgent need for developing systems able to personalize the online experience of Web users on the basis of their needs. Web recommendation is a promising technology that attempts to predict the interests of Web users, by providing them with information and/or services that they need without explicitly asking for them. In this paper we propose NEWER, a usage-based Web recommendation system that exploits the potential of Computational Intelligence techniques to dynamically suggest interesting pages to users according to their preferences. NEWER employs a neuro-fuzzy approach in order to determine categories of users sharing similar interests and to discover a recommendation model as a set of fuzzy rules expressing the associations between user categories and relevances of pages. The discovered model is used by a online recommendation module to determine the list of links judged relevant for users. The results obtained on both synthetic and real-world data show that NEWER is effective for recommendation, leading to a quality of the generated recommendations comparable and often significantly better than those of other approaches employed for the comparison.

A fuzzy clustering approach for mining diagnostic rules

In this paper an approach for automatic discovery of transparent diagnostic rules from data is proposed. The approach is based on a fuzzy clustering technique that is defined by three sequential steps. First, our Crisp Double Clustering algorithm is applied on available symptoms measurements, to provide a set of representative multidimensional prototypes that are further clustered onto each one-dimensional projection. The resulting clusters are used in the second step, where a set of fuzzy relations are defined in terms of transparent fuzzy sets. As a final step, the derived fuzzy relations are employed to define a set of fuzzy rules, which establish the knowledge base of a fuzzy inference system that can be used for fuzzy diagnosis. The approach has been applied to the Aachen Aphasia dataset as a real-world benchmark and compared with related work.