Oswaldo Ludwig

Novel Maximum-Margin Training Algorithms for Supervised Neural Networks

This paper proposes three novel training methods, two of them based on the backpropagation approach and a third one based on information theory for multilayer perceptron (MLP) binary classifiers. Both backpropagation methods are based on the maximal-margin (MM) principle. The first one, based on the gradient descent with adaptive learning rate algorithm (GDX) and named maximum-margin GDX (MMGDX), directly increases the margin of the MLP output-layer hyperplane. The proposed method jointly optimizes both MLP layers in a single process, backpropagating the gradient of an MM-based objective function, through the output and hidden layers, in order to create a hidden-layer space that enables a higher margin for the output-layer hyperplane, avoiding the testing of many arbitrary kernels, as occurs in case of support vector machine (SVM) training. The proposed MM-based objective function aims to stretch out the margin to its limit. An objective function based on Lp-norm is also proposed in order to take into account the idea of support vectors, however, overcoming the complexity involved in solving a constrained optimization problem, usually in SVM training. In fact, all the training methods proposed in this paper have time and space complexities O(N) while usual SVM training methods have time complexity O(N 3) and space complexity O(N 2) , where N is the training-data-set size. The second approach, named minimization of interclass interference (MICI), has an objective function inspired on the Fisher discriminant analysis. Such algorithm aims to create an MLP hidden output where the patterns have a desirable statistical distribution. In both training methods, the maximum area under ROC curve (AUC) is applied as stop criterion. The third approach offers a robust training framework able to take the best of each proposed training method. The main idea is to compose a neural model by using neurons extracted from three other neural networks, each one previously trained by MICI, MMGDX, and Levenberg-Marquard (LM), respectively. The resulting neural network was named assembled neural network (ASNN). Benchmark data sets of real-world problems have been used in experiments that enable a comparison with other state-of-the-art classifiers. The results provide evidence of the effectiveness of our methods regarding accuracy, AUC, and balanced error rate.

Exploiting LIDAR-based features on pedestrian detection in urban scenarios

Reliable detection and classification of vulnerable road users constitute a critical issue on safety/protection systems for intelligent vehicles driving in urban zones. In this subject, most of the perception systems have LIDAR and/or Radar as primary detection modules and vision-based systems for object classification. This work, on the other hand, presents a valuable analysis of pedestrian detection in urban scenario using exclusively LIDAR-based features. The aim is to explore how much information can be extracted from LIDAR sensors for pedestrian detection. Moreover, this study will be useful to compose multi-sensor based pedestrian detection systems using not only LIDAR but also vision sensors. Experimental results using our data set and a detailed classification performance analysis are presented, with comparisons among various classification techniques.

Improving the Generalization Properties of Neural Networks: an Application to Vehicle Detection

In this paper a multilayer feedforward neural network based approach for vehicle detection is proposed. The main idea is to use such network to perform both feature extraction and classification. This simplicity enables real time applications. In order to achieve such capabilities, the network is trained by a new algorithm, proposed in this paper, named minimization of inter-class interference (MCI). Such algorithm aims to create a hidden space (i.e. feature space) where the patterns have a desirable statistical distribution. Regarding the neural architecture, the linear output layer is replaced by the Mahalanobis kernel, in order to improve generalization. Experiments are performed by means of a dataset that includes two standard datasets from Caltech car rear. Finally, disturbed images are used, in order to evaluate the robustness of the neural-network based vehicle detection. The proposed method reveals low miss rate, low false alarm rate and high area under ROC curve. In Matlab environment, the algorithm spends only 3.280e-4 seconds per image. These facts encourage this research line.

A cascade classifier applied in pedestrian detection using laser and image-based features

In this paper we present a multistage method applied in pedestrian detection using information from a LIDAR and a monocular-camera mounted on an electric vehicle driving in urban scenarios. The proposed method is a cascade of classifiers trained in two subsets of features, one with laser-based features and the other with a set of image-based features. A specific training approach was developed to adjust the cascade stages in order to enhance the classification performance. The proposed method differs from the conventional cascade regarding the way the selected samples are propagated through the cascade. Thus, the subsequent stages of the proposed cascade receive both negatives and positives from previous ones, relying on a decision margin process. Experiments were conducted in off-line mode, for a set of single component classifiers and for the proposed cascade technique. The results are compared in terms of classification performance metrics and ROC curves.

Eigenvalue decay: A new method for neural network regularization

This paper proposes two new training algorithms for multilayer perceptrons based on evolutionary computation, regularization, and transduction. Regularization is a commonly used technique for preventing the learning algorithm from overfitting the training data. In this context, this work introduces and analyzes a novel regularization scheme for neural networks (NNs) named eigenvalue decay, which aims at improving the classification margin. The introduction of eigenvalue decay led to the development of a new training method based on the same principles of SVM, and so named Support Vector NN (SVNN). Finally, by analogy with the transductive SVM (TSVM), it is proposed a transductive NN (TNN), by exploiting SVNN in order to address transductive learning. The effectiveness of the proposed algorithms is evaluated on seven benchmark datasets.

Design of Automatic Meter Reading based on Zigbee

This paper describes the study about Automatic Meter Reading (AMR) in indoor environments, implementing a WSN (Wireless Sensor Network) based on Zigbee technology. Automatic Meter Reading is used for remote collection of the utilities data. And these utilities may means electricity, gas, water consumption data or any other. Our concentration will be on Electricity power monitoring system which can monitor power quality, can remotely control power service which will enable prepaid billing.

Evaluation of Boosting-SVM and SRM-SVM cascade classifiers in laser and vision-based pedestrian detection

Pedestrian detection systems constitute an important field of research and development in computer vision, specially when applied in protection/safety systems in urban scenarios due to their direct impact in the society, specifically in terms of traffic casualties. In order to face such challenge, this work exploits some developments on statistical machine learning theory, in particular structural risk minimization (SRM) in a cascade ensemble. Namely, the ensemble applies the principle of SRM on a set of linear support vector machines (SVM). The linear SVM complexity, in the Vapnik sense, is controlled by choosing the dimension of the feature space in each cascade stage. To support experimental analysis, a multi-sensor dataset constituted by data from a LIDAR, a monocular camera, an IMU, encoder and a DGPS is introduced in this paper. The dataset, named Laser and Image Pedestrian Detection (LIPD) dataset, was collected in an urban environment, at day light conditions, using an electrical vehicle driven at low speed. Labeled pedestrians and non-pedestrians samples are also available for benchmarking purpose. The cascade of SVMs, trained with image-based features (HOG and COV descriptors), is used to detect pedestrian evidences on regions of interest (ROI) generated by a LIDAR-based processing system. Finally, the paper presents experimental results comparing the performance of a Boosting-SVM cascade and the proposed SRM-SVM cascade classifiers, in terms of detection errors.

Parking spaces detection in indoor environments based on Zigbee

This paper describes the study about parking spaces detection in indoor environments, implementing a WSN (Wireless Sensor Network) based on Zigbee technology. Our study focuses on the analysis of the received power when a vehicle is parked or not, by monitoring the RSSI (Received Signal Strength Indicator) field defined in the MAC layer as the IEEE 802.15.4 describes. We focus in the analysis of three factors: data processing time, the influence of distance in received power and the attenuation effect in the signal when propagates, which determine the network topology to deploy and the validity of our location model.

Improving the Generalization Capacity of Cascade Classifiers

The cascade classifier is a usual approach in object detection based on vision, since it successively rejects negative occurrences, e.g., background images, in a cascade structure, keeping the processing time suitable for on-the-fly applications. On the other hand, similar to other classifier ensembles, cascade classifiers are likely to have high Vapnik-Chervonenkis (VC) dimension, which may lead to overfitting the training data. Therefore, this work aims at improving the generalization capacity of the cascade classifier by controlling its complexity, which depends on the model of their classifier stages, the number of stages, and the feature space dimension of each stage, which can be controlled by integrating the parameter setting of the feature extractor (in our case an image descriptor) into the maximum-margin framework of support vector machine training, as will be shown in this paper. Moreover, to set the number of cascade stages, bounds on the false positive rate (FP) and on the true positive rate (TP) of cascade classifiers are derived based on a VC-style analysis. These bounds are applied to compose an enveloping receiver operating curve (EROC), i.e., a new curve in the TP-FP space in which each point is an ordered pair of upper bound on the FP and lower bound on the TP. The optimal number of cascade stages is forecasted by comparing EROCs of cascades with different numbers of stages.

Entropy analysis applied to NFIR models

This present work has been developed in view of a research project that aims a mapping process between infrared images and thermo pair sensors readings in diesel motors by the use of artificial neural network. The consistence analysis of a set of examples used in the supervised training of artificial neural networks is presented. The proposed approach is based on the entropy analysis of both input and output data. The principal component analysis method is applied to avoid redundant information that can lead to an entropy level overestimation.