Anna Maria Fanelli

An iterative pruning algorithm for feedforward neural networks

The problem of determining the proper size of an artificial neural network is recognized to be crucial, especially for its practical implications in such important issues as learning and generalization. One popular approach for tackling this problem is commonly known as pruning and it consists of training a larger than necessary network and then removing unnecessary weights/nodes. In this paper, a new pruning method is developed, based on the idea of iteratively eliminating units and adjusting the remaining weights in such a way that the network performance does not worsen over the entire training set. The pruning problem is formulated in terms of solving a system of linear equations, and a very efficient conjugate gradient algorithm is used for solving it, in the least-squares sense. The algorithm also provides a simple criterion for choosing the units to be removed, which has proved to work well in practice. The results obtained over various test problems demonstrate the effectiveness of the proposed approach.

Interpretability constraints for fuzzy information granulation

Information granules are complex entities that arise in the process of abstraction of data and derivation of knowledge. The automatic generation of information granules from data is an important task, since it gives to machines the ability of acquiring knowledge that can be communicated to users. For this purpose, knowledge acquisition should provide for fuzzy information granules that can be naturally labelled by linguistic terms, i.e. symbols that belong to the natural language. Information granules of this type are called interpretable. However, interpretability cannot be guaranteed until a set of constraints is imposed on the granulation process. In literature, several interpretability constraints have been proposed, but, due to the subjective interpretation of interpretability, there is no agreement on which constraints should be adopted. This survey is an attempt to provide for a complete presentation of interpretability constraints adopted in literature with the following objectives: (i) to give a homogeneous description of all interpretability constraints; (ii) to provide for a critical review of such constraints; (iii) to identify potentially different meanings of interpretability. Hopefully, this survey may serve as a guidance for designing interpretable fuzzy models as well as for identifying new methods of interpretable information granulation.

Variable selection using neural-network models

Abstract In this paper we propose an approach to variable selection that uses a neural-network model as the tool to determine which variables are to be discarded. The method performs a backward selection by successively removing input nodes in a network trained with the complete set of variables as inputs. Input nodes are removed, along with their connections, and remaining weights are adjusted in such a way that the overall input–output behavior learnt by the network is kept approximately unchanged. A simple criterion to select input nodes to be removed is developed. The proposed method is tested on a famous example of system identification. Experimental results show that the removal of input nodes from the neural network model improves its generalization ability. In addition, the method compares favorably with respect to other feature reduction methods.

Computer-aided simulation for bone surgery

A system for evaluating bone deformities using a 3-D model directly recovered from 2-D images and for simulating surgery is described. It derives a 3-D object representation from only two X-ray images. It also offers user-friendly simulation of bone surgery with low-cost hardware and software. The system exhibits satisfactory behavior for reconstructing the bone shape, providing suitable data for the simulation and evaluation of bone surgery. Although the spline interpolation of the bone surface does not produce a realistic 3-D visualization of the tibia, which is used as an example, the reconstruction is useful in solving problems inherent in the pathology considered.<<ETX>>

Interpretability assessment of fuzzy knowledge bases

Computing with words (CWW) relies on linguistic representation of knowledge that is processed by operating at the semantical level defined through fuzzy sets. Linguistic representation of knowledge is a major issue when fuzzy rule based models are acquired from data by some form of empirical learning. Indeed, these models are often requested to exhibit interpretability, which is normally evaluated in terms of structural features, such as rule complexity, properties on fuzzy sets and partitions. In this paper we propose a different approach for evaluating interpretability that is based on the notion of cointension. The interpretability of a fuzzy rule-based model is measured in terms of cointension degree between the explicit semantics, defined by the formal parameter settings of the model, and the implicit semantics conveyed to the reader by the linguistic representation of knowledge. Implicit semantics calls for a representation of users knowledge which is difficult to externalise. Nevertheless, we identify a set of properties -- which we call “logical view” -- that is expected to hold in the implicit semantics and is used in our approach to evaluate the cointension between explicit and implicit semantics. In practice, a new fuzzy rule base is obtained by minimising the fuzzy rule base through logical properties. Semantic comparison is made by evaluating the performances of the two rule bases, which are supposed to be similar when the two semantics are almost equivalent. If this is the case, we deduce that the logical view is applicable to the model, which can be tagged as interpretable from the cointension viewpoint. These ideas are then used to define a strategy for assessing interpretability of fuzzy rule-based classifiers (FRBCs). The strategy has been evaluated on a set of pre-existent FRBCs, acquired by different learning processes from a well-known benchmark dataset. Our analysis highlighted that some of them are not cointensive with users knowledge, hence their linguistic representation is not appropriate, even though they can be tagged as interpretable from a structural point of view.

Knowledge discovery by a neuro-fuzzy modeling framework

In this paper a neuro-fuzzy modeling framework is proposed, which is devoted to discover knowledge from data and represent it in the form of fuzzy rules. The core of the framework is a knowledge extraction procedure that is aimed to identify the structure and the parameters of a fuzzy rule base, through a two-phase learning of a neuro-fuzzy network. In order to obtain reliable and readable knowledge, two further stages are integrated with the knowledge extraction procedure: a pre-processing stage, performing variable selection on the available data to obtain simpler and more reliable fuzzy rules, and a post-processing stage, that granulates outputs of the extracted fuzzy rules so as to provide a validity range of estimated outputs. Moreover, the framework can address complex multi-input multi-output problems. In such case, two distinct modeling strategies can be followed with the opportunity of producing both a single MIMO model or a collection of MISO models. The proposed framework is verified on a real-world case study, involving prediction of chemical composition of ashes produced by combustion processes carried out in thermo-electric generators located in Italy.

Distinguishability quantification of fuzzy sets

Distinguishability is a semantic property of fuzzy sets that has a great relevance in the design of interpretable fuzzy models. Distinguishability has been mathematically defined through different measures, which are addressed in this paper. Special emphasis is given to similarity, which exhibits sound theoretical properties but its calculation is usually computationally intensive, and possibility, whose calculation can be very efficient but it does not exhibit the same properties of similarity. It is shown that under mild conditions – usually met in interpretable fuzzy modeling – possibility can be used as a valid measure for assessing distinguishability, thus overcoming the computational inefficiencies of similarity measures. Moreover, procedures that minimize possibility also minimize similarity and, consequently, improve distinguishability. In this sense, the use of possibility is fully justified in interpretable fuzzy modeling.

A neuro-fuzzy network to generate human-understandable knowledge from data

Neuro-fuzzy networks have been successfully applied to extract knowledge from data in the form of fuzzy rules. However, one drawback with the neuro-fuzzy approach is that the fuzzy rules induced by the learning process are not necessarily understandable. The lack of readability is essentially due to the high dimensionality of the parameter space that leads to excessive flexibility in the modification of parameters during learning. In this paper, to obtain readable knowledge from data, we propose a new neuro-fuzzy model and its learning algorithm that works in a parameter space with reduced dimensionality. The dimensionality of the new parameter space is necessary and sufficient to generate human-understandable fuzzy rules, in the sense formally defined by a set of properties. The learning procedure is based on a gradient descent technique and the proposed model is general enough to be applied to other neuro-fuzzy architectures. Simulation studies on a benchmark and a real-life problem are carried out to embody the idea of the paper.

Meta-data: characterization of input features for meta-learning

Common inductive learning strategies offer the tools for knowledge acquisition, but possess some inherent limitations due to the use of fixed bias during the learning process. To overcome limitations of such base-learning approaches, a novel research trend is oriented to explore the potentialities of meta-learning, which is oriented to the development of mechanisms based on a dynamical search of bias. This could lead to an improvement of the base-learner performance on specific learning tasks, by profiting of the accumulated past experience. As a significant set of I/O data is needed for efficient base-learning, appropriate meta-data characterization is of crucial importance for useful meta-learning. In order to characterize meta-data, firstly a collection of meta-features discriminating among different base-level tasks should be identified. This paper focuses on the characterization of meta-data, through an analysis of meta-features that can capture the properties of specific tasks to be solved at base level. This kind of approach represents a first step toward the development of a meta-learning system, capable of suggesting the proper bias for base-learning different specific task domains.

Recent advances in artificial neural networks: design and applications

A Neuro-Symbolic Hybrid Intelligent Architecture With Applications. New Radical Basis Neural Networks and Their Application In A Large-Scale Handwritten Digit Recognition Problem. Efficient Neural Network-Based Methodology for the Design of Multiple Classifiers. Learning Fine Motion in Robotics: Design and Experiments. A New Neural Network For Adaptive Pattern Recognition Of Multichannel Input Signals. Lateral Priming Adaptive Resonance Theory (LAPART)-2: Innovation in Art. Neural Network Learning in A Travel Reservation Domain. Recent Advances In Neural Network Applications in Process Control. Monitoring Internal Combustion Engines by Neural Network Based Virtual Sensing. Neural Architectures of Fuzzy Petri Nets. Index. Each Chapter Includes an Introduction and References. NTI/Sales Copy