Mingyu You

Emotion Recognition from Noisy Speech

This paper presents an emotion recognition system from clean and noisy speech. Geodesic distance was adopted to preserve the intrinsic geometry of emotional speech. Based on the geodesic distance estimation, an enhanced Lipschitz embedding was developed to embed the 64-dimensional acoustic features into a six-dimensional space. In order to avoid the problems brought by noise reduction, emotion recognition from noisy speech was performed directly. Linear discriminant analysis (LDA), principal component analysis (PCA) and feature selection by sequential forward selection (SFS) with support vector machine (SVM) were also included to compress acoustic features before classifying the emotional states of clean and noisy speech. Experimental results demonstrate that compared with other methods, the proposed system makes approximately 10% improvement. The performance of our system is also robust when speech data is corrupted by increasing noise

Speech Emotion Recognition using an Enhanced Co-Training Algorithm

In previous systems of speech emotion recognition, supervised learning are frequently employed to train classifiers on lots of labeled examples. However, the labeling of abundant data requires much time and many human efforts. This paper presents an enhanced co-training algorithm to utilize a large amount of unlabeled speech utterances for building a semi-supervised learning system. It uses two conditionally independent attribute views(i.e. temporal features and statistic features) of unlabeled examples to augment a much smaller set of labeled examples. Our experimental results demonstrate that compared with the method based on the supervised training, the proposed system makes 9.0% absolute improvement on female model and 7.4% on male model in terms of average accuracy. Moreover, the enhanced co-training algorithm achieves comparable performance to the co-training prototype, while it can reduce the classification noise which is produced by error labeling in the process of semi-supervised learning.

Emotional Speech Analysis on Nonlinear Manifold

This paper presents a speech emotion recognition system on nonlinear manifold. Instead of straight-line distance, geodesic distance was adopted to preserve the intrinsic geometry of speech corpus. Based on geodesic distance estimation, we developed an enhanced Lipschitz embedding to embed the 64-dimensional acoustic features into a six-dimensional space. In this space, speech data with the same emotional state were located close to one plane, which was beneficial to emotion classification. The compressed testing data were classified into six archetypal emotional states (neutral, anger, fear, happiness, sadness and surprise) by a trained linear support vector machine (SVM) system. Experimental results demonstrate that compared with traditional methods of feature extraction on linear manifold and feature selection, the proposed system makes 9%-26% relative improvement in speaker-independent emotion recognition and 5%-20% improvement in speaker-dependent

A Hierarchical Framework for Speech Emotion Recognition

Dimensionality reduction is an important issue in pattern recognition. Two popular methods used in this field are principal component analysis (PCA) and linear discriminant analysis (LDA). In this paper, detailed comparisons were performed among PCA, LDA and PCA+LDA considering the lack of similar studies. It showed that no particular method was optimal across all emotion categories. Based on this analysis, a new framework combining PCA and LDA was proposed. An appropriate dimensionality reduction method was employed for every emotion category in the new framework. Experimental results demonstrate that our approach achieves a better overall performance compared with PCA, LDA or PCA+LDA

Speech Emotion Recognition Based on a Fusion of All-Class and Pairwise-Class Feature Selection

Traditionally in speech emotion recognition, feature selection(FS) is implemented by considering the features from all classes jointly. In this paper, a hybrid system based on all-class FS and pairwise-class FS is proposed to improve speech emotion classification performance. Besides a subset of features obtained from an all-class structure, FS is performed on the available data from each pair of classes. All these subsets are used in their corresponding K-nearest-neighbors(KNN) or Support Vector Machine(SVM) classifiers and the posterior probabilities of the multi-classifiers are fused hierarchically. The experiment results demonstrate that compared with the classical method based on all-class FS and the pairwise method based on pairwise-class FS, the proposed approach achieves 3.2%-8.4% relative improvement on the average F1-measure in speaker-independent emotion recognition.

An enhanced speech emotion recognition system based on discourse information

There are certain correlation between two persons’ emotional states in communication, but none of previous work has focused on it. In this paper, a novel conversation database in Chinese was collected and an emotion interaction matrix was proposed to embody the discourse information in conversation. Based on discourse information, an enhanced speech emotion recognition system was presented to improve the recognition accuracy. Some modifications were performed on traditional KNN classification, which could reduce the interruption of noise. Experiment result shows that our system makes 3% – 5% relative improvement compared with the traditional method.

Speech Emotion Recognition and Intensity Estimation

In this paper, a system for speech emotion analysis is presented. On a corpus of over 1700 utterances from an individual, the feature vector stream is extracted for each utterance based on short time log frequency power coefficients (LFCC). Using the feature vector streams, we trained Hidden Markov Models (HMMs) to recognize seven basic categories emotions: neutral, happiness, anger, sadness, surprise, fear. Furthermore, the intensity of the basic emotion is divided into 3 levels. And we trained 18 sub-HMMs to identify the intensity of the recognized emotions. Experiment result shows that the emotion recognition rate and the estimation of intensity performed by our system are of good and convincing quality.

CHAD: a chinese affective database

Affective database plays an important role in the process of affective computing which has been an attractive field of AI research. Based on analyzing current databases, a Chinese affective database (CHAD) is designed and established for seven emotion states: neutral, happy, sad, fear, angry, surprise and disgust. Instead of choosing the personal suggestion method, audiovisual materials are collected in four ways including three types of laboratory recording and movies. Broadcast programmes are also included as source of vocal corpus. By comparison of the five sources two points are gained. First, although broadcast programmes get the best performance in listening experiment, there are still problems as copyright, lacking visual information and can not represent the characteristics of speech in daily life. Second, laboratory recording using sentences with appropriately emotional content is an outstanding source of materials which has a comparable performance with broadcasts.

A Novel Classifier Based on Enhanced Lipschitz Embedding for Speech Emotion Recognition

The paper proposes a novel classifier named ELEC (Enhanced Lipschitz Embedding based Classifier) in the speech emotion recognition system. ELEC adopts geodesic distance to preserve the intrinsic geometry of speech corpus and embeds the high dimensional feature vector into a lower space. Through analyzing the class labels of the neighbor training vectors in the compressed space, ELEC classifies the test data into six archetypal emotional states, i.e. neutral, anger, fear, happiness, sadness and surprise. Experimental results on a benchmark data set demonstrate that compared with the traditional methods, the proposed classifier of ELEC achieves 17% improvement in average for speaker-independent emotion recognition and 13% for speaker-dependent.

Facial animation system for embedded application

This paper describes a prototype implementation of a speech driven facial animation system for embedded devices. The system is comprised of speech recognition and talking head synthesis. A context-based visubsyllable database is set up to map Chinese initials or finals to their corresponding pronunciation mouth shape. With the database, 3D facial animation can be synthesized based on speech signal input. Experiment results show the system works well in simulating real mouth shapes and forwarding a friendly interface in communication terminals.