Liyanage C. De Silva

Speech emotion recognition using hidden Markov models

Abstract In emotion classification of speech signals, the popular features employed are statistics of fundamental frequency, energy contour, duration of silence and voice quality. However, the performance of systems employing these features degrades substantially when more than two categories of emotion are to be classified. In this paper, a text independent method of emotion classification of speech is proposed. The proposed method makes use of short time log frequency power coefficients (LFPC) to represent the speech signals and a discrete hidden Markov model (HMM) as the classifier. The emotions are classified into six categories. The category labels used are, the archetypal emotions of Anger, Disgust, Fear, Joy, Sadness and Surprise. A database consisting of 60 emotional utterances, each from twelve speakers is constructed and used to train and test the proposed system. Performance of the LFPC feature parameters is compared with that of the linear prediction Cepstral coefficients (LPCC) and mel-frequency Cepstral coefficients (MFCC) feature parameters commonly used in speech recognition systems. Results show that the proposed system yields an average accuracy of 78% and the best accuracy of 96% in the classification of six emotions. This is beyond the 17% chances by a random hit for a sample set of 6 categories. Results also reveal that LFPC is a better choice as feature parameters for emotion classification than the traditional feature parameters.

Ensemble methods for spoken emotion recognition in call-centres

Machine-based emotional intelligence is a requirement for more natural interaction between humans and computer interfaces and a basic level of accurate emotion perception is needed for computer systems to respond adequately to human emotion. Humans convey emotional information both intentionally and unintentionally via speech patterns. These vocal patterns are perceived and understood by listeners during conversation. This research aims to improve the automatic perception of vocal emotion in two ways. First, we compare two emotional speech data sources: natural, spontaneous emotional speech and acted or portrayed emotional speech. This comparison demonstrates the advantages and disadvantages of both acquisition methods and how these methods affect the end application of vocal emotion recognition. Second, we look at two classification methods which have not been applied in this field: stacked generalisation and unweighted vote. We show how these techniques can yield an improvement over traditional classification methods.

State of the art of smart homes

In this paper we present a review of the state of the art of smart homes. We will first look at the research work related to smart homes from various view points; first in the view point of specific techniques such as smart homes that utilize computer vision based techniques, smart homes that utilize audio-based techniques and then smart homes that utilize multimodal techniques. Then we look at it from the view point of specific applications of smart homes such as eldercare and childcare applications, energy efficiency applications and finally in the research directions of multimedia retrieval for ubiquitous environments. We will summarize the smart homes based research into these two categories. In the survey we found out that some well-known smart home applications like video based security applications has seen the maturity in terms of new research directions while some topics like smart homes for energy efficiency and video summarization are gaining momentum.

Detection and tracking of facial features

Detection and tracking of facial features without using any head mounted devices may become required in various future visual communication applications, such as teleconferencing, virtual reality etc. In this paper we propose an automatic method of face feature detection using a method called edge pixel counting. Instead of utilizing color or gray scale information of the facial image, the proposed edge pixel counting method utilized the edge information to estimate the face feature positions such as eyes, nose and mouth in the first frame of a moving facial image sequence, using a variable size face feature template. For the remaining frames, feature tracking is carried out alternatively using a method called deformable template matching and edge pixel counting. One main advantage of using edge pixel counting in feature tracking is that it does not require the condition of a high inter frame correlation around the feature areas as is required in template matching. Some experimental results are shown to demonstrate the effectiveness of the proposed method.

nPlug: a smart plug for alleviating peak loads

The Indian electricity sector, despite having the worlds fifth largest installed capacity, suffers from a 12.9% peaking shortage. This shortage could be alleviated, if a large number of deferrable loads, particularly the high powered ones, could be moved from on-peak to off-peak times. However, conventional DSM strategies may not be suitable for India as the local conditions usually favor only inexpensive solutions with minimal dependence on the pre-existing infrastructure. In this work, we present nPlug, a smart plug that sits between the wall socket and deferrable loads such as water heaters, washing machines, and electric vehicles. nPlugs combine real-time sensing and analytics to infer peak periods as well as supply-demand imbalance and reschedule attached appliances in a decentralized manner to alleviate peaks whenever possible. They do not require any manual intervention by the end consumer nor any enhancements to the appliances or existing infrastructure. Some of nPlugs capabilities are demonstrated using experiments on a combination of synthetic and real data collected from plug-level energy monitors. Our results indicate that nPlug can be an effective and inexpensive technology to address the peaking shortage.

Cloud basis function neural network: A modified RBF network architecture for holistic facial expression recognition

The paper presents novel modifications to radial basis functions (RBFs) and a neural network based classifier for holistic recognition of the six universal facial expressions from static images. The new basis functions, called cloud basis functions (CBFs) use a different feature weighting, derived to emphasize features relevant to class discrimination. Further, these basis functions are designed to have multiple boundary segments, rather than a single boundary as for RBFs. These new enhancements to the basis functions along with a suitable training algorithm allow the neural network to better learn the specific properties of the problem domain. The proposed classifiers have demonstrated superior performance compared to conventional RBF neural networks as well as several other types of holistic techniques used in conjunction with RBF neural networks. The CBF neural network based classifier yielded an accuracy of 96.1%, compared to 86.6%, the best accuracy obtained from all other conventional RBF neural network based classification schemes tested using the same database.

A hybrid approach of NN and HMM for facial emotion classification

Neural networks (NNs) are often combined with Hidden Markov Models (HMMs) in speech recognition for achieving superior performance. In this paper, this hybrid approach is employed in facial emotion classification. Gabor wavelets are employed to extract features from difference images obtained by subtracting the first frame showing a frontal face from the current frame. The NN, which takes the form of Multilayer perceptron (MLP), is used to classify the feature vector into different states of a HMM of a certain emotion sequence, i.e., neutral, intermediate and peak. In addition to using 1-0 as targets for the NN, a heuristic strategy of assigning variable targets 1-x-0 has also been applied. After training, we interpret the output values of the NN as the posterior of the HMM state and directly apply the Viterbi algorithm to these values to estimate the best state path. The experiments show that with variable targets for the NN, the HMM gives better results than that with 1-0 targets. The best HMM results are obtained for x = 0.8 in 1-x-0.

Voting ensembles for spoken affect classification

Affect or emotion classification from speech has much to benefit from ensemble classification methods. In this paper we apply a simple voting mechanism to an ensemble of classifiers and attain a modest performance increase compared to the individual classifiers. A natural emotional speech database was compiled from 11 speakers. Listener-judges were used to validate the emotional content of the speech. Thirty-eight prosody-based features correlating characteristics of speech with emotional states were extracted from the data. A classifier ensemble was designed using a multi-layer perceptron, support vector machine, K* instance-based learner, K-nearest neighbour, and random forest of decision trees. A simple voting scheme determined the most popular prediction. The accuracy of the ensemble is compared with the accuracies of the individual classifiers.

Modeling and Recognition of Gesture Signals in 2D Space: A Comparison of NN and SVM Approaches

In this paper we introduce a novel technique for modeling and recognizing gesture signals in 2D space. This technique is based on measuring the direction of the gradient of the movement trajectory as features of the gesture signal. Each gesture signal is represented as a time series of gradient angle values. These features are classified by applying a given classification method. In this article we compared the accuracy of a feed forward artificial neural network with a support vector machine using a radial kernel. The comparison was based on the recorded data of 13 gesture signals as training and testing data. The average accuracy of the ANN and SVM were 98.27% and 96.34% respectively. The false detection ratio was 3.83% for ANN and 8.45% for SVM, which suggests the ANN is more suitable for gesture signal recognition

Multi-sensor based human activity detection for smart homes

At the University of Brunei Darussalam, we have designed and built a prototype smart home to monitor human activities to improve the energy efficiency and support elder people. In this paper we present some of our early work related to smart monitoring, control and communication along with a review of other related research initiatives by researchers around the world. Especially we looked at research work carried out in Singapore, Japan and New Zealand. Here our main objective was to look into research work that enhances energy efficiency and eldercare with the use of multitude of sensors. With our simple prototype implementation we have also demonstrated the use of smart home technologies to reduce energy consumption in an average house.