Bruno Apolloni

From synapses to rules

We consider an integrated subsymbolic-symbolic procedure for extracting symbolically explained classification rules from data. A multilayer perceptron maps features into propositional variables and a set of subsequent layers operated by a PAC-like algorithm learns boolean expressions on these variables. The peculiarities of the whole procedure are: (i) we do not know a priori the class of formulas these expressions belong to, rather from time to time we get some information about the class and reduce uncertainty about the current hypothesis; (ii) the mapping from features to variables varies also over time to improve the suitability of the desired classification rules; and (iii) the final shape of the learnt expressions is determined by the learner who can express his preferences both in terms of an error function to be backpropagated along all layers of the proposed architecture and through the choice of a set of free parameters. We review the bases of the first point and then analyze the others in depth. The theoretical tools supporting the analysis are: (1) a new statistical framework that we call algorithmic inference; (2) a special functionality of the sampled points in respect to the formulas, denoted sentineling; and (3) entropy measures and fuzzy set methods governing the whole learning process. Preliminary numerical results highlight the value of the procedure.

Quantum stochastic optimization

We propose a combinatorial optimization procedure based on the physical idea of using the quantum tunnel effect to allow the search of global minima of a function of many Boolean variables to escape from poor local minima. More specifically, the function V to be minimized is viewed as the potential energy term in a Schrodinger Hamiltonian H for a quantum spin 1/2 system, the kinetic energy term being the generator of a random walk tailored to the neighborhood structure associated with V The distorted random walk associated with (a suitable approximation of) the ground state eigenfunction of H defines then our approximate optimization strategy. A numerical application to the graph partitioning problem is presented.

On emotion recognition of faces and of speech using neural networks, fuzzy logic and the ASSESS system

We propose a framework for the processing of face image sequences and speech, using different dynamic techniques to extract appropriate features for emotion recognition. The features will be used by a hybrid classification procedure, employing neural network techniques and fuzzy logic, to accumulate the evidence for the presence of an emotional expression of the face and the speakers voice.

Learning fuzzy decision trees

We present a recurrent neural network which learns to suggest the next move during the descent along the branches of a decision tree. More precisely, given a decision instance represented by a node in the decision tree, the network provides the degree of membership of each possible move to the fuzzy set z.Lt;good movez.Gt;. These fuzzy values constitute the core of the probability of selecting the move out of the set of the children of the current node.This results in a natural way for driving the sharp discrete-state process running along the decision tree by means of incremental methods on the continuous-valued parameters of the neural network. The bulk of the learning problem consists in stating useful links between the local decisions about the next move and the global decisions about the suitability of the final solution. The peculiarity of the learning task is that the network has to deal explicitly with the twofold charge of lighting up the best solution and generating the move sequence that leads to that solution. We tested various options for the learning procedure on the problem of disambiguating natural language sentences.

Simulated annealing approach in backpropagation

Abstract Looking at the training stage of error-backpropagation algorithm as an optimization problem, in this paper we check two ways of embedding simulated annealing to improve the usual gradient descent method for the achievement of good minima of the error function. The first way refers to a continuous state multilayer perceptron (MLP) and it stands for a random selection of the descent direction around the steepest one. The second way concerns binary states MLP where a backpropagation of the right answer from output to input is realized through a Boltzmann machine.

PAC learning of concept classes through the boundaries of their items

Abstract We present a new perspective for investigating the probably approximate correct (PAC) learnability of classes of concepts. We focus on special sets of points for characterizing the concepts within their class. This gives rise to a general notion of boundary of a concept, which holds even in discrete spaces, and to a special probability measuring technique. This technique is applied (i) to narrow the gap between the minimum and maximum sample sizes necessary to learn even under a more stringent learnability definition, and (ii) to get self-explanatory indices of the complexity of the learning task. These indices can be roughly estimated during the learning process and appear very useful in the treatment of nonsymbolic procedures, e.g. in the context of neutral networks.

Reputation features for trust prediction in social networks

Trust prediction in Social Networks is required to solve the cold start problem, which consists of guessing a Trust value when the truster has no direct previous experience on the trustee. Trust prediction can be achieved by the application of machine learning approaches applied to reputation features, which are extracted from the available Trust information provided by witness users. Conventional machine learning methods work on a fixed dimension space, so that variable size reputation information must be reduced to a fixed size vector. We propose and give validation results on two approaches, (1) a naive selection of reputation features, and (2) a probabilistic model of these features. We report experimental results on trust prediction over publicly available Epinions and Wikipedia adminship voting databases achieving encouraging results.

A co-operating neural approach for spacecrafts attitude control

Abstract A locally recurrent neural network is described as a key component of a control system able to rule an artificial satellite whose attitude must be kept close to zero-angle with respect to an inertial reference system earth centred. The main idea is to join a simple linear adaptive controller with a neural network trained to compensate the inadequacy of the former. The control signal is the sum of the signal computed by the two devices; the feedback for training the neural network comes from the attitude error w.r.t. a reference trajectory and is computed by means of a linear inversion of the satellite dynamics. Thanks to such co-operation, the resulting system is easily trainable and performs efficiently. In fact, the whole system acts as a MRAC controller whose accuracy has been tested on numerical simulations of an Olympus class spacecraft. Considerations on stability, reactions to unexpected solicitations, extension to non-geocentric missions and power consumption are included as well.

Granular regression

We augment a linear regression procedure by a thruth-functional method in order to identify a highly informative regression line. The idea is to use statistical methods to identify a confidence region for the line and exploit the structure of the sample data falling in this region for identifying the most fitting line. The fitness function is related to the fuzziness of the sampled points as a natural extension of the statistical criterion ruling the identification of the confidence region within the Algorithmic Inference approach. We tested the method on three well known benchmarks.

Feature selection via Boolean independent component analysis

We devise a feature selection method in terms of a follow-out utility of a special classification procedure. In turn, we root the latter on binary features which we extract from the input patterns with a wrapper method. The whole contrivance results in a procedure that is progressive in two respects. As for features, first we compute a very essential representation of them in terms of Boolean independent components in order to reduce their entropy. Then we reverse the representation mapping to discover the subset of the original features supporting a successful classification. As for the classification, we split it into two less hard tasks. With the former we look for a clustering of input patterns that satisfies loose consistency constraints and benefits from the conciseness of binary representation. With the latter we attribute labels to the clusters through the combined use of basically linear separators. We implement out the method through a relatively quick numerical procedure by assembling a set of connectionist and symbolic routines. These we toss on the benchmark of feature selection of DNA microarray data in cancer diagnosis and other ancillary datasets.