João Roberto Bertini

A nonparametric classification method based on K-associated graphs

Graph is a powerful representation formalism that has been widely employed in machine learning and data mining. In this paper, we present a graph-based classification method, consisting of the construction of a special graph referred to as K-associated graph, which is capable of representing similarity relationships among data cases and proportion of classes overlapping. The main properties of the K-associated graphs as well as the classification algorithm are described. Experimental evaluation indicates that the proposed technique captures topological structure of the training data and leads to good results on classification task particularly for noisy data. In comparison to other well-known classification techniques, the proposed approach shows the following interesting features: (1) A new measure, called purity, is introduced not only to characterize the degree of overlap among classes in the input data set, but also to construct the K-associated optimal graph for classification; (2) nonlinear classification with automatic local adaptation according to the input data. Contrasting to K-nearest neighbor classifier, which uses a fixed K, the proposed algorithm is able to automatically consider different values of K, in order to best fit the corresponding overlap of classes in different data subspaces, revealing both the local and global structure of input data. (3) The proposed classification algorithm is nonparametric, implicating high efficiency and no need for model selection in practical applications.

Constructive Neural Network Algorithms for Feedforward Architectures Suitable for Classification Tasks

This chapter presents and discusses several well-known constructive neural network algorithms suitable for constructing feedforward architectures aiming at classification tasks involving two classes. The algorithms are divided into two different groups: the ones directed by the minimization of classification errors and those based on a sequential model. In spite of the focus being on two-class classification algorithms, the chapter also briefly comments on the multiclass versions of several two-class algorithms, highlights some of the most popular constructive algorithms for regression problems and refers to several other alternative algorithms.

An incremental learning algorithm based on the K-associated graph for non-stationary data classification

Non-stationary classification problems concern the changes on data distribution over a classifier lifetime. To face this problem, learning algorithms must conciliate essential, but difficult to gather, attributes like good classification performance, stability and low associated costs, like processing time and memory. This paper presents an extension of the K-associated optimal graph learning algorithm to cope with classification over non-stationary domains. The algorithm relies on a graph structure consisting of many disconnected components (subgraphs). Such graph enhances data representation by fitting locally groups of data according to a purity measure, which, in turn, quantifies the overlapping between vertices of different classes. As a result, the graph can be used to accurately estimate the probability of unlabeled data to belong to a given class. The proposed algorithm is benefited from the dynamical evolution of the graph by updating its set of components when new data is presented along time, by removing old components as new components arise. Experimental results on artificial and real domains and further statistical analysis show that the proposed algorithm is an effective solution to non-stationary classification problems.

Chaotic synchronization in general network topology for scene segmentation

Chaotic synchronization has been discovered to be an important property of neural activities, which in turn has encouraged many researchers to develop chaotic neural networks for scene and data analysis. In this paper, we study the synchronization role of coupled chaotic oscillators in networks of general topology. Specifically, a rigorous proof is presented to show that a large number of oscillators with arbitrary geometrical connections can be synchronized by providing a sufficiently strong coupling strength. Moreover, the results presented in this paper not only are valid to a wide class of chaotic oscillators, but also cover the parameter mismatch case. Finally, we show how the obtained result can be applied to construct an oscillatory network for scene segmentation.

Classification Based on the Optimal K-Associated Network

In this paper, we propose a new graph-based classifier which uses a special network, referred to as optimal K-associated network, for modeling data. The K-associated network is capable of representing (dis)similarity relationships among data samples and data classes. Here, we describe the main properties of the K-associated network as well as the classification algorithm based on it. Experimental evaluation indicates that the model based on an optimal K-associated network captures topological structure of the training data leading to good results on the classification task particularly for noisy data.

Ensemble of complete P-partite graph classifiers for non-stationary environments

Non-stationary data can be characterized as data having a distribution that changes over time. It is well-known that most successful machine learning algorithms are based on stationary data i.e., data that are assumed to have a fixed distribution (although unknown, in most cases). Non-stationary classification problems require the induced classifiers to be flexible enough to learn or adapt themselves to reflect the changes on data distribution over time; this can be a hard task, taking into account that changes that may happen are not usually known in advance. Although there are several proposals in the literature that deal with non-stationary data, none of them deal with missing attribute values, a common problem in real applications. This paper proposes an ensemble of classifiers for non-stationary environments that (1) uses a new graph structure for representing data known as Complete P-partite Attribute-based Decision Graph - CPp-AbDG; (2) handles data described by heterogeneous attributes (numeric and categorical) and (3) handles missing attribute values. Experiments in non-stationary environments show evidence of the strength of the CPp-AbDG representation as well as the potentiality of the proposed ensemble approach.

MBabCoNN --- A Multiclass Version of a Constructive Neural Network Algorithm Based on Linear Separability and Convex Hull

The essential characteristic of constructive neural network (CoNN) algorithms is the incremental construction of the neural network architecture along with the training process. The BabCoNN (Barycentric-based constructive neural network) algorithm is a new neural network constructive algorithm suitable for two-class problems that relies on the BCP (Barycentric Correction Procedure) for training its individual TLU (Threshold Logic Unit). Motivated by the good results obtained with the two-class BabCoNN, this paper proposes its extension to multiclass domains as a new CoNN algorithm named MBabCoNN. Besides describing the main concepts involved in the MBabCoNN proposal, the paper also presents a comparative analysis of its performance versus the multiclass versions of five well known constructive algorithms, in four knowledge domains as an empirical evidence of the MBabCoNN suitability and efficiency for multiclass classification tasks.

A Comparative Evaluation of Constructive Neural Networks Methods using PRM and BCP as TLU Training Algorithms

Constructive neural network algorithms enable the architecture of a neural network to be constructed as an intrinsic part of the learning process. These algorithms are very dependent on the TLU training algorithm they employ. Generally they use a Perceptron-based algorithm (such as Pocket or Pocket with Ratchet Modification (PRM)) for training each individual node added to the network, during the learning process. In the literature can be found a vast selection of algorithms for training individual TLUs. This paper investigates the use of the Barycentric Correction Procedure (BCP) algorithm with four constructive algorithms namely Tower, Pyramid, Shift and Perceptron-Cascade. Results show that some constructive neural algorithms have better performance using BCP than using PRM.

Refining constructive neural networks using functionally expanded input data

This paper reports an empirical investigation on the use of functionally expanded input data for the constructive learning of neural networks; a functional expansion can be helpful when approximating nonlinear functions. The investigation was conducted considering six constructive neural network algorithms (Tower, Pyramid, Tiling, PTI, Perceptron Cascade and Shift), six data domains (four real data and two artificial) and two polynomial expansions (power set series and trigonometric). Results from experiments conducted are presented; a comparative analysis is given as evidence of the benefits of functionally expanding the input data, as a pre-processing step prior to learning, when constructive neural network algorithms are used.

Imputation of missing data supported by Complete p-Partite attribute-based Decision Graphs

Missing attribute values is a recurrent problem in data mining and machine learning. Although there are plenty of techniques to handle this problem, most of them are too simplistic to provide a good estimation for absent attribute values. A very active research area focuses on solving the missing attribute value problem via imputation methods, which replaces missing data with substituted values. This paper proposes a new imputation method which uses a special graph named Complete p-Partite Attribute-based Decision Graphs (CpP-AbDG) to estimate, in a consistent and plausible way, the missing values. The graph is built by considering the range of each attribute that describes the data divided into sub-intervals; sub-intervals are approached as the vertices of a graph. Edges are then established between pairs of different vertices, provided they do not related to the same attribute. The edges and vertices are finally assigned a weight, based on distributions of the classes. The resulting CpP-AbDG has shown to be a suitable and informative data structure for finding the proper interval in which a missing attribute value should lie, taking into account all the attributes that describe the data. Results comparing the proposed approach to classical ones in an computational environment that considers classification problems as an evaluation criteria, show the potential of the method.