Lele Cao

Building feature space of extreme learning machine with sparse denoising stacked-autoencoder

The random-hidden-node extreme learning machine (ELM) is a much more generalized cluster of single-hidden-layer feed-forward neural networks (SLFNs) which has three parts: random projection, non-linear transformation, and ridge regression (RR) model. Networks with deep architectures have demonstrated state-of-the-art performance in a variety of settings, especially with computer vision tasks. Deep learning algorithms such as stacked autoencoder (SAE) and deep belief network (DBN) are built on learning several levels of representation of the input. Beyond simply learning features by stacking autoencoders (AE), there is a need for increasing its robustness to noise and reinforcing the sparsity of weights to make it easier to discover interesting and prominent features. The sparse AE and denoising AE was hence developed for this purpose. This paper proposes an approach: SSDAE-RR (stacked sparse denoising autoencoder - ridge regression) that effectively integrates the advantages in SAE, sparse AE, denoising AE, and the RR implementation in ELM algorithm. We conducted experimental study on real-world classification (binary and multiclass) and regression problems with different scales among several relevant approaches: SSDAE-RR, ELM, DBN, neural network (NN), and SAE. The performance analysis shows that the SSDAE-RR tends to achieve a better generalization ability on relatively large datasets (large sample size and high dimension) that were not pre-processed for feature abstraction. For 16 out of 18 tested datasets, the performance of SSDAE-RR is more stable than other tested approaches. We also note that the sparsity regularization and denoising mechanism seem to be mandatory for constructing interpretable feature representations. The fact that a SSDAE-RR approach often has a comparable training time to ELM makes it useful in some real applications.

Sparse Coding and Dictionary Learning with Linear Dynamical Systems

Linear Dynamical Systems (LDSs) are the fundamental tools for encoding spatio-temporal data in various disciplines. To enhance the performance of LDSs, in this paper, we address the challenging issue of performing sparse coding on the space of LDSs, where both data and dictionary atoms are LDSs. Rather than approximate the extended observability with a finite-order matrix, we represent the space of LDSs by an infinite Grassmannian consisting of the orthonormalized extended observability subspaces. Via a homeomorphic mapping, such Grassmannian is embedded into the space of symmetric matrices, where a tractable objective function can be derived for sparse coding. Then, we propose an efficient method to learn the system parameters of the dictionary atoms explicitly, by imposing the symmetric constraint to the transition matrices of the data and dictionary systems. Moreover, we combine the state covariance into the algorithm formulation, thus further promoting the performance of the models with symmetric transition matrices. Comparative experimental evaluations reveal the superior performance of proposed methods on various tasks including video classification and tactile recognition.

A Deep and Stable Extreme Learning Approach for Classification and Regression

The random-hidden-node based extreme learning machine (ELM) is a much more generalized cluster of single-hidden-layer feed-forward neural networks (SLFNs) whose hidden layer do not need to be adjusted, and tends to reach both the smallest training error and the smallest norm of output weights. Deep belief networks (DBNs) are probabilistic generative modals composed of simple, unsupervised networks such as restricted Boltzmann machines (RBMs) or auto-encoders, where each sub-network’s hidden layer serves as the visible layer for the next. This paper proposes an approach: DS-ELM (a deep and stable extreme learning machine) that combines a DBN with an ELM. The performance analysis on real-world classification (binary and multi-category) and regression problems shows that DS-ELM tends to achieve a better performance on relatively large datasets (large sample size and high dimension). In most tested cases, DS-ELM’s performance is generally more stable than ELM and DBN in solving classification problems. Moreover, the training time consumption of DS-ELM is comparable to ELM.

A new slip-detection method based on pairwise high frequency components of capacitive sensor signals

In this paper, a novel method for slip detection using a capacitive sensor is proposed. We perform the Discrete Wavelet Transform (DWT) on the original signals of sensor. By comparing different wavelets, we find that the Haar wavelet is the most suitable to separate different frequency components. After performing the DWT by using the Haar wavelet, the separated high frequency components are pairwise due to properties of the Haar wavelet. Different from setting thresholds to detect object slip, our method detects slip by observing the variation trend of pairwise high frequency components. Meanwhile, we can distinguish signals of object loading and slip respectively. We carry out experiments on several objects with different surface properties and the results are consistent with our observations.

Tactile sequence based object categorization: A Bag of features modeled by Linear Dynamic System with Symmetric Transition Matrix.

In this paper, we propose a novel categorization framework to recognize tactile sequences based on two particular properties of the tactile data. For the first one, tactile sequences are spatio-temporal data which is sequential and dynamic, depicting the process of grasping an object in different grasping stages; therefore, it is reasonable to discover the dynamical pattern by modeling tactile data as integral sequences rather than individual frames. For the second one, a tactile sequence contains various dynamical patterns in different stages of the grasping process; therefore, we decompose the whole sequence into multiple mini-sequences so as to enhance feature resolution. To address both properties in our framework, we take advantage of a Bag-of-System model using parameters of the Linear Dynamic System (LDS) as feature descriptors. Moreover, we employ the LDS with Symmetric Transition matrix (LDSST) rather than the original LDS as the building-block in order to obtain accurate codewords of the codebook of the Bag-of-System. The performance of our framework is evaluated on six real-world databases of three groups. Our experiments show that classification using LDSST is better than the original LDS, and the decomposition of tactile sequences does improve the accuracy of classification. The experiment results also show the superiority of our framework in comparison with other state-of-the-art sequence classifiers.

Real-Time Recurrent Tactile Recognition: Momentum Batch-Sequential Echo State Networks

Tactile recognition aims at identifying target objects according to tactile sensory readings. Tactile data have two salient properties: 1) sequentially real-time and 2) temporally correlated, which essentially calls for a real-time (i.e., online fixed-budget) and recurrent recognition procedure. Based on an efficient and robust spatio-temporal feature representation for tactile sequences, we handle the problem of real-time recurrent tactile recognition by proposing a bounded online-sequential learning framework, and incorporates the strength of batch-regularization bootstrapping, bounded recursive reservoir, and momentum-based estimation. Experimental evaluations show that it outperforms the state-of-the-art methods by a large margin on test accuracy; and its training performance is superior to most compared models from aspects of average online training error, computational complexity, and storage efficiency.

Advancing the incremental fusion of robotic sensory features using online multi-kernel extreme learning machine

Robot recognition tasks usually require multiple homogeneous or heterogeneous sensors which intrinsically generate sequential, redundant, and storage demanding data with various noise pollution. Thus, online machine learning algorithms performing efficient sensory feature fusion have become a hot topic in robot recognition domain. This paper proposes an online multi-kernel extreme learning machine (OM-ELM) which assembles multiple ELM classifiers and optimizes the kernel weights with a p-norm formulation of multi-kernel learning (MKL) problem. It can be applied in feature fusion applications that require incremental learning over multiple sequential sensory readings. The performance of OM-ELM is tested towards four different robot recognition tasks. By comparing to several state-of-the-art online models for multi-kernel learning, we claim that our method achieves a superior or equivalent training accuracy and generalization ability with less training time. Practical suggestions are also given to aid effective online fusion of robot sensory features.