Bailing Zhang

Multiresolution forecasting for futures trading using wavelet decompositions

We investigate the effectiveness of a financial time-series forecasting strategy which exploits the multiresolution property of the wavelet transform. A financial series is decomposed into an over complete, shift invariant scale-related representation. In transform space, each individual wavelet series is modeled by a separate multilayer perceptron (MLP). We apply the Bayesian method of automatic relevance determination to choose short past windows (short-term history) for the inputs to the MLPs at lower scales and long past windows (long-term history) at higher scales. To form the overall forecast, the individual forecasts are then recombined by the linear reconstruction property of the inverse transform with the chosen autocorrelation shell representation, or by another perceptron which learns the weight of each scale in the prediction of the original time series. The forecast results are then passed to a money management system to generate trades.

Handwritten digit recognition by adaptive-subspace self-organizing map (ASSOM)

The adaptive-subspace self-organizing map (ASSOM) proposed by Kohonen is a recent development in self-organizing map (SOM) computation. In this paper, we propose a method to realize ASSOM using a neural learning algorithm in nonlinear autoencoder networks. Our method has the advantage of numerical stability. We have applied our ASSOM model to build a modular classification system for handwritten digit recognition. Ten ASSOM modules are used to capture different features in the ten classes of digits. When a test digit is presented to all the modules, each module provides a reconstructed pattern and the system outputs a class label by comparing the ten reconstruction errors. Our experiments show promising results. For relatively small size modules, the classification accuracy reaches 99.3% on the training set and over 97% on the testing set.

Gabor wavelet associative memory for face recognition

This letter describes a high-performance face recognition system by combining two recently proposed neural network models, namely Gabor wavelet network (GWN) and kernel associative memory (KAM), into a unified structure called Gabor wavelet associative memory (GWAM). GWAM has superior representation capability inherited from GWN and consequently demonstrates a much better recognition performance than KAM. Extensive experiments have been conducted to evaluate a GWAM-based recognition scheme using three popular face databases, i.e., FERET database, Olivetti-Oracle Research Lab (ORL) database and AR face database. The experimental results consistently show our schemes superiority and demonstrate its very high-performance comparing favorably to some recent face recognition methods, achieving 99.3% and 100% accuracy, respectively, on the former two databases, exhibiting very robust performance on the last database against varying illumination conditions.

Reliable Classification of Vehicle Types Based on Cascade Classifier Ensembles

Vehicle-type recognition based on images is a challenging task. This paper comparatively studied two feature extraction methods for image description, i.e., the Gabor wavelet transform and the Pyramid Histogram of Oriented Gradients (PHOG). The Gabor transform has been widely adopted to extract image features for various vision tasks. PHOG has the superiority in its description of more discriminating information. A highly reliable classification scheme was proposed by cascade classifier ensembles with reject option to accommodate the situations where no decision should be made if there exists adequate ambiguity. The first ensemble is heterogeneous, consisting of several classifiers, including k-nearest neighbors (kNNs), multiple-layer perceptrons (MLPs), support vector machines (SVMs), and random forest. The classification reliability is further enhanced by a second classifier ensemble, which is composed of a set of base MLPs coordinated by an ensemble metalearning method called rotation forest (RF). For both of the ensembles, rejection option is accomplished by relating the consensus degree from majority voting to a confidence measure and by abstaining to classify ambiguous samples if the consensus degree is lower than a threshold. The final class label is assigned by dual majority voting from the two ensembles. Experimental results using more than 600 images from a variety of 21 makes of cars and vans demonstrated the effectiveness of the proposed approach. The cascade ensembles produce consistently reliable results. With a moderate ensemble size of 25 in the second ensemble, the two-stage classification scheme offers 98.65% accuracy with a rejection rate of 2.5%, exhibiting promising potential for real-world applications.

A nonlinear neural network model of mixture of local principal component analysis: application to handwritten digits recognition

Abstract Principal component analysis (PCA) is a popular tool in multivariate statistics and pattern recognition. Recently, some mixture models of local principal component analysis have attracted attention due to a number of benefits over global PCA. In this paper, we propose a mixture model by concurrently performing global data partition and local linear PCA. The partition is optimal or near optimal, which is realized by a soft competition algorithm called ‘neural gas’. The local PCA type representation is approximated by a neural learning algorithm in a nonlinear autoencoder network, which is set up on the generalization of the least-squares reconstruction problem leading to the standard PCA. Such a local PCA type representation has a number of numerical advantages, for example, faster convergence and insensitive to local minima. Based on this mixture model, we describe a modular classification scheme to solve the problem of handwritten digits recognition. We use 10 networks (modules) to capture different features in the 10 classes of handwritten digits, with each network being a mixture model of local PCA type representations. When a test digit is presented to all the modules, each module provides a reconstructed pattern by a prescribed principle and the system outputs the class label by comparing the reconstruction errors from the 10 networks. Compared with some traditional neural network-based classifiers, our scheme converges faster and recognizes with higher accuracy. For a relatively small size of each module, the classification accuracy reaches 98.6% on the training set and 97.8% on the testing set.

A kernel autoassociator approach to pattern classification

Autoassociators are a special type of neural networks which, by learning to reproduce a given set of patterns, grasp the underlying concept that is useful for pattern classification. In this paper, we present a novel nonlinear model referred to as kernel autoassociators based on kernel methods. While conventional non-linear autoassociation models emphasize searching for the non-linear representations of input patterns, a kernel autoassociator takes a kernel feature space as the nonlinear manifold, and places emphasis on the reconstruction of input patterns from the kernel feature space. Two methods are proposed to address the reconstruction problem, using linear and multivariate polynomial functions, respectively. We apply the proposed model to novelty detection with or without novelty examples and study it on the promoter detection and sonar target recognition problems. We also apply the model to mclass classification problems including wine recognition, glass recognition, handwritten digit recognition, and face recognition. The experimental results show that, compared with conventional autoassociators and other recognition systems, kernel autoassociators can provide better or comparable performance for concept learning and recognition in various domains.

One-class kernel subspace ensemble for medical image classification

Classification of medical images is an important issue in computer-assisted diagnosis. In this paper, a classification scheme based on a one-class kernel principle component analysis (KPCA) model ensemble has been proposed for the classification of medical images. The ensemble consists of one-class KPCA models trained using different image features from each image class, and a proposed product combining rule was used for combining the KPCA models to produce classification confidence scores for assigning an image to each class. The effectiveness of the proposed classification scheme was verified using a breast cancer biopsy image dataset and a 3D optical coherence tomography (OCT) retinal image set. The combination of different image features exploits the complementary strengths of these different feature extractors. The proposed classification scheme obtained promising results on the two medical image sets. The proposed method was also evaluated on the UCI breast cancer dataset (diagnostic), and a competitive result was obtained.

Gait classification in children with cerebral palsy by Bayesian approach

Cerebral palsy (CP) is generally considered as a nonprogressive neuro-developmental condition that occurs in early childhood and is associated with a motor impairment, usually affecting mobility and posture. Automatic accurate identification of cerebral palsy gait has many potential applications, for example, assistance in diagnosis, clinical decisionmaking and communication among the clinical professionals. In previous studies, support vector machine (SVM) and neural networks have been applied to classify CP gait patterns. However, one of the disadvantages of SVM and many neural network models is that given a gait sample, it only predicts a gait pattern class label without providing any estimate of the underlying probability, which is particularly important in Computer Aided Diagnostics applications. The objective of this study is to first investigate different pattern classification paradigms in the automatic gait analysis and address the significance of Bayesian classifier model, and then give a comprehensive performances comparison. Using a publicly available CP gait dataset (68 normal healthy and 88 with spastic diplegia form of CP), different features including the two basic temporal-spatial gait parameters (stride length and cadence) have been experimented. Various hold-out and cross-validation testing show that the Bayesian model offers excellent classification performances compared with some popular classifiers such as random forest and multiple layer perceptron. With many advantages considered, Bayesian classifier model is very significant in establishing a clinical decision system for gait analysis.

Phenotype Recognition with Combined Features and Random Subspace Classifier Ensemble

BackgroundAutomated, image based high-content screening is a fundamental tool for discovery in biological science. Modern robotic fluorescence microscopes are able to capture thousands of images from massively parallel experiments such as RNA interference (RNAi) or small-molecule screens. As such, efficient computational methods are required for automatic cellular phenotype identification capable of dealing with large image data sets. In this paper we investigated an efficient method for the extraction of quantitative features from images by combining second order statistics, or Haralick features, with curvelet transform. A random subspace based classifier ensemble with multiple layer perceptron (MLP) as the base classifier was then exploited for classification. Haralick features estimate image properties related to second-order statistics based on the grey level co-occurrence matrix (GLCM), which has been extensively used for various image processing applications. The curvelet transform has a more sparse representation of the image than wavelet, thus offering a description with higher time frequency resolution and high degree of directionality and anisotropy, which is particularly appropriate for many images rich with edges and curves. A combined feature description from Haralick feature and curvelet transform can further increase the accuracy of classification by taking their complementary information. We then investigate the applicability of the random subspace (RS) ensemble method for phenotype classification based on microscopy images. A base classifier is trained with a RS sampled subset of the original feature set and the ensemble assigns a class label by majority voting.ResultsExperimental results on the phenotype recognition from three benchmarking image sets including HeLa, CHO and RNAi show the effectiveness of the proposed approach. The combined feature is better than any individual one in the classification accuracy. The ensemble model produces better classification performance compared to the component neural networks trained. For the three images sets HeLa, CHO and RNAi, the Random Subspace Ensembles offers the classification rates 91.20%, 98.86% and 91.03% respectively, which compares sharply with the published result 84%, 93% and 82% from a multi-purpose image classifier WND-CHARM which applied wavelet transforms and other feature extraction methods. We investigated the problem of estimation of ensemble parameters and found that satisfactory performance improvement could be brought by a relative medium dimensionality of feature subsets and small ensemble size.ConclusionsThe characteristics of curvelet transform of being multiscale and multidirectional suit the description of microscopy images very well. It is empirically demonstrated that the curvelet-based feature is clearly preferred to wavelet-based feature for bioimage descriptions. The random subspace ensemble of MLPs is much better than a number of commonly applied multi-class classifiers in the investigated application of phenotype recognition.

Off-Line Signature Recognition and Verification by Kernel Principal Component Self-Regression

Automatic signature verification is an active area of research with numerous applications such as bank check verification, ATM access, etc. In this research, a kernel principal component self-regression (KPCSR) model is proposed for offline signature verification and recognition problems. Developed from the kernel principal component regression (KPCR), the self-regression model selects a subset of the principal components from the kernel space for the input variables to accurately characterize each users signature, thus offering good verification and recognition performance. The model directly works on bitmap images in the preliminary experiments, showing satisfactory performance. A modular scheme with subject-specific KPCSR structure proves very efficient, from which each user is assigned an independent KPCSR model for coding the corresponding visual information. Experimental results obtained on public benchmarking signature databases demonstrate the superiority of the proposed method