Tianchi Liu

Discriminative clustering via extreme learning machine

Discriminative clustering is an unsupervised learning framework which introduces the discriminative learning rule of supervised classification into clustering. The underlying assumption is that a good partition (clustering) of the data should yield high discrimination, namely, the partitioned data can be easily classified by some classification algorithms. In this paper, we propose three discriminative clustering approaches based on Extreme Learning Machine (ELM). The first algorithm iteratively trains weighted ELM (W-ELM) classifier to gradually maximize the data discrimination. The second and third methods are both built on Fishers Linear Discriminant Analysis (LDA); but one approach adopts alternative optimization, while the other leverages kernel k-means. We show that the proposed algorithms can be easily implemented, and yield competitive clustering accuracy on real world data sets compared to state-of-the-art clustering methods.

Extreme Learning Machine for Clustering

Extreme Learning Machine (ELM) is originally introduced for regression and classification. This paper extends ELM for clustering using Extreme Learning Machine Auto Encoder (ELM-AE) which learn key features of the input data. The embedding created by multiplying the input data with the output weights of ELM-AE is shown to produce better clustering results than clustering the original data space. Furthermore, ELM-AE is used to find the starting cluster points for k-means clustering, which produces better results than randomly assigning the cluster start points. The experimental results show that the proposed clustering algorithm Extreme Learning Machine Auto Encoder Clustering (ELM-AEC) is better than k-means clustering and is competitive with Unsupervised Extreme Learning Machine (USELM).

Smile detection using Pair-wise Distance Vector and Extreme Learning Machine

A smile is a common human facial expression used as the indicator for positive emotion. The detection of smiling has many applications, for example, controlling camera shutter when a smile is detected and measuring the degree of satisfaction during a video conference. Many feature extraction methods have been proposed for detecting a smile in the unconstrained scenarios. However, the dimensions of most existing feature descriptors are too huge, which limits their real applications. Moreover, features should be more effective to distinguish between smile face and non-smile face. Motivated by the observation that the mouth shape can effectively reflect a persons smile state, we extracted a novel and snappy set of features that form a feature vector named Pair-wise Distance Vector, which is calculated only based on few points around a mouth. After that, Extreme Learning Machine (ELM) is adopted to classify smile based on these features. The experimental results on GENKI-4K database show that our proposed method outperforms the state-of-the-art methods in terms of accuracy and dimension of features.

Detection of Drivers’ Distraction Using Semi-Supervised Extreme Learning Machine

Monitoring drivers’ visual behavior using machine learning techniques has been identified as an effective approach to detect and mitigate driver distraction to enhance road safety. In our previous work, detection system based on supervised Extreme Learning Machine (ELM) was developed and tested with satisfactory performance. However, supervised ELM requires all training data to be labeled, which can be costly and time-consuming. This paper proposed and evaluated a semi-supervised distraction detection system based on Semi-Supervised Extreme Learning Machine (SS-ELM). The experimental results show that SS-ELM outperformed supervised ELM in both accuracy (95.5% for SS-ELM vs. 93.0% for ELM) and model sensitivity (97.6% for SS-ELM and 95.5% for ELM), suggesting that the proposed semi-supervised detection system can extract information from unlabeled data effectively to improve the performance. SS-ELM based detection system has the potential of improving accuracy and alleviating the cost of adapting distraction detection systems to new drivers, and thus is more promising for real world applications.

Driver Workload Detection in On-Road Driving Environment Using Machine Learning

Drivers’ high workload caused by distractions has become one of the major concerns for road safety. This paper presents a data-driven method using machine learning algorithms to detect high workload caused by surrogate in-vehicle (IV) secondary tasks performed in an on-road experiment with real traffic. The data were collected using an instrumented vehicle while drivers performed two types of secondary tasks: visual-manual and auditory-vocal tasks. Two types of machine learning methods, support vector machine (SVM) and extreme learning machine (ELM), were applied to detect drivers’ workload via drivers’ visual behaviour (i.e. eye movements) data alone, as well as visual plus driving performance data. The results suggested that both methods can detect drivers’ workload at high accuracy, with ELM outperformed SVM in most cases. We found that for visual intensive workload, using drivers’ visual data alone achieveed an accuracy close to using the combination information from both visual and driving performance data. This study proves that machine learning methods can be used for real driving applications.

Cluster Regularized Extreme Learning Machine for Detecting Mixed-Type Distraction in Driving

Distraction was previously studied within each dimension separately, i.e., physical, cognitive and visual. However real-world activities usually involve multiple distraction dimensions in terms of brain resources that might conflict with the driving task. This brings difficulties for classifying dimension/type of distraction even for human experts. On the other hand, many subsequent functional blocks do not utilize distraction type information. For example, a pre-collision system usually makes decision based on distraction level rather than distraction type. Therefore this study aims to detect distraction in general regardless of its type, and proposes an effective machine learning algorithm, i.e., Cluster Regularized Extreme Learning Machine (CR-ELM), to detect mixed-type distraction in driving. Compared to traditional machine learning techniques, CR-ELM is designed to handle problems with multiple clusters per class, and provides more accurate detection performance, which could be used for advanced driver assistance systems.

Manifold ranking based scoring system with its application to cardiac arrest prediction

BACKGROUND The recently developed geometric distance scoring system has shown the effectiveness of scoring systems in predicting cardiac arrest within 72h and the potential to predict other clinical outcomes. However, the geometric distance scoring system predicts scores based on only local structure embedded by the data, thus leaving much room for improvement in terms of prediction accuracy. METHODS We developed a novel scoring system for predicting cardiac arrest within 72h. The scoring system was developed based on a semi-supervised learning algorithm, manifold ranking, which explores both the local and global consistency of the data. System evaluation was conducted on emergency department patients׳ data, including both vital signs and heart rate variability (HRV) parameters. Comparison of the proposed scoring system with previous work was given in terms of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV). RESULTS Out of 1025 patients, 52 (5.1%) met the primary outcome. Experimental results show that the proposed scoring system was able to achieve higher area under the curve (AUC) on both the balanced dataset (0.907 vs. 0.824) and the imbalanced dataset (0.774 vs. 0.734) compared to the geometric distance scoring system. CONCLUSIONS The proposed scoring system improved the prediction accuracy by utilizing the global consistency of the training data. We foresee the potential of extending this scoring system, as well as manifold ranking algorithm, to other medical decision making problems. Furthermore, we will investigate the parameter selection process and other techniques to improve performance on the imbalanced dataset.

Two-stage structured learning approach for stable occupancy detection.

Monitoring the presence of occupants in a room in a timely manner is a fundamental step for effective building management. Environmental sensor networks have the advantages of high cost-efficiency and non-intrusiveness on privacy and are very suitable for room occupancy detection. Nonlinear discriminative models, e.g., support vector machine and neural networks, have shown good detection performance due to their ability to model complex relationship. However, they tend to produce unstable detection with frequent fluctuations over time, because they regard training data as independent and ignore the prior knowledge of the room occupancy, i.e., not changing very frequently. To improve the stability of the detection, we propose a two-stage structured learning approach with Extreme Learning Machine (ELM) as the local classifier. In the first stage, ELM is used as a fast nonlinear classifier to obtain preliminary detection results. In the second stage, we form data sequences consisting of the current and previous data points. The preliminary detection results by ELM of the data sequences are then used as input to a linear support vector machine for structured output to generate the final detection results. We test the proposed two-stage structured learning approach on a real-world dataset and show that the proposed approach outperforms the related machine learning methods.

ELM based smile detection using Distance Vector

Abstract The Smile is one of the most common facial expressions, and it serves as an indicator of the positive emotion. Many feature extraction methods have been proposed for detecting a smile in an unconstrained scene. However, most of the existing feature descriptors are too large and not effective to be applied to distinguish smile and non-smile in the real world. In this paper, we proposed an ELM-based smile detection system by using a novel feature extraction method. Motivated by the observation that the mouth shape can effectively reflect a person’s smile state, a novel and snappy set of features from a few of facial landmarks around the mouth are extracted. We have tested our algorithms on the smile detection database, and the results indicate that our method is better than the state-of-the-art methods with higher accuracy and lower dimension of features.

Effects of two new features of approximate entropy and sample entropy on cardiac arrest prediction

Sixteen conventional heart beat variability (HRV) parameters and eight vital signs have shown promise in the prediction of cardiac arrest within 72 hours. Besides these 24 parameters, we proposed adding two new features for cardiac arrest prediction, which are approximate entropy (ApEn) and sample entropy (SpEn). ApEn and SpEn are nonlinear HRV parameters capable of characterizing heart conditions. These two entropies were derived from electrocardiography recordings and combined with the existing 24 features to form feature combinations. The experiments were conducted by using linear kernel Support Vector Machine classification technique to investigate the effects of using ApEn, SpEn together with 24 parameters on cardiac arrest prediction. The dimensionality reduction approach, Principal Component Analysis, was applied to suppress the dimensionality. Results reveal that the prediction performance of adding ApEn and SpEn to the 24 parameters is improved significantly compared to using the 24 parameters only. Dimension reduction has additional positive effects on improving the prediction results.