Constantine Kotropoulos

Emotional speech recognition: Resources, features, and methods

In this paper we overview emotional speech recognition having in mind three goals. The first goal is to provide an up-to-date record of the available emotional speech data collections. The number of emotional states, the language, the number of speakers, and the kind of speech are briefly addressed. The second goal is to present the most frequent acoustic features used for emotional speech recognition and to assess how the emotion affects them. Typical features are the pitch, the formants, the vocal tract cross-section areas, the mel-frequency cepstral coefficients, the Teager energy operator-based features, the intensity of the speech signal, and the speech rate. The third goal is to review appropriate techniques in order to classify speech into emotional states. We examine separately classification techniques that exploit timing information from which that ignore it. Classification techniques based on hidden Markov models, artificial neural networks, linear discriminant analysis, k-nearest neighbors, support vector machines are reviewed.

Using support vector machines to enhance the performance of elastic graph matching for frontal face authentication

A novel method for enhancing the performance of elastic graph matching in frontal face authentication is proposed. The starting point is to weigh the local similarity values at the nodes of an elastic graph according to their discriminatory power. Powerful and well-established optimization techniques are used to derive the weights of the linear combination. More specifically, we propose a novel approach that reformulates Fishers discriminant ratio to a quadratic optimization problem subject to a set of inequality constraints by combining statistical pattern recognition and support vector machines (SVM). Both linear and nonlinear SVM are then constructed to yield the optimal separating hyperplanes and the optimal polynomial decision surfaces, respectively. The method has been applied to frontal face authentication on the M2VTS database. Experimental results indicate that the performance of morphological elastic graph matching is highly improved by using the proposed weighting technique.

Automatic emotional speech classification

Our purpose is to design a useful tool which can be used in psychology to automatically classify utterances into five emotional states such as anger, happiness, neutral, sadness, and surprise. The major contribution of the paper is to rate the discriminating capability of a set of features for emotional speech recognition. A total of 87 features has been calculated over 500 utterances from the Danish Emotional Speech database. The sequential forward selection method (SFS) has been used in order to discover a set of 5 to 10 features which are able to classify the utterances in the best way. The criterion used in SFS is the cross-validated correct classification score of one of the following classifiers: nearest mean and Bayes classifier where class pdf are approximated via Parzen windows or modelled as Gaussians. After selecting the 5 best features, we reduce the dimensionality to two by applying principal component analysis. The result is a 51.6% /spl plusmn/ 3% correct classification rate at 95% confidence interval for the five aforementioned emotions, whereas a random classification would give a correct classification rate of 20%. Furthermore, we find out those two-class emotion recognition problems whose error rates contribute heavily to the average error and we indicate that a possible reduction of the error rates reported in this paper would be achieved by employing two-class classifiers and combining them.

Rule-based face detection in frontal views

Face detection is a key problem in building automated systems that perform face recognition. A very attractive approach for face detection is based on multiresolution images (also known as mosaic images). Motivated by the simplicity of this approach, a rule-based face detection algorithm in frontal views is developed that extends the work of G. Yang and T.S. Huang (see Pattern Recognition, vol.27, no.1, p.53-63, 1994). The proposed algorithm has been applied to frontal views extracted from the European ACTS M2VTS database that contains the videosequences of 37 different persons. It has been found that the algorithm provides a correct facial candidate in all cases. However, the success rate of the detected facial features (e.g. eyebrows/eyes, nostrils/nose, and mouth) that validate the choice of a facial candidate is found to be 86.5% under the most strict evaluation conditions.

Frontal face authentication using morphological elastic graph matching

A novel dynamic link architecture based on multiscale morphological dilation-erosion is proposed for frontal face authentication. Instead of a set of Gabor filters tuned to different orientations and scales, multiscale morphological operations are employed to yield a feature vector at each node of the reference grid. Linear projection algorithms for feature selection and automatic weighting of the nodes according to their discriminatory power succeed to increase the authentication capability of the method. The performance of the morphological dynamic link architecture is evaluated in terms of the receiver operating characteristic in the M2VTS face image database. The comparison with other frontal face authentication algorithms indicates that the morphological dynamic link architecture with discriminatory power coefficients is the best algorithm with respect to the equal error rate achieved.

ICA and Gabor representation for facial expression recognition

Two hybrid systems for classifying seven categories of human facial expression are proposed. The first system combines independent component analysis (ICA) and support vector machines (SVMs). The original face image database is decomposed into linear combinations of several basis images, where the corresponding coefficients of these combinations are fed up into SVMs instead of an original feature vector comprised of grayscale image pixel values. The classification accuracy of this system is compared against that of baseline techniques that combine ICA with either two-class cosine similarity classifiers or two-class maximum correlation classifiers, when we classify facial expressions into these seven classes. We found that, ICA decomposition combined with SVMs outperforms the aforementioned baseline classifiers. The second system proposed operates in two steps: first, a set of Gabor wavelets (GWs) is applied to the original face image database and, second, the new features obtained are classified by using either SVMs or cosine similarity classifiers or maximum correlation classifier. The best facial expression recognition rate is achieved when Gabor wavelets are combined with SVMs.

Fast and accurate sequential floating forward feature selection with the Bayes classifier applied to speech emotion recognition

This paper addresses subset feature selection performed by the sequential floating forward selection (SFFS). The criterion employed in SFFS is the correct classification rate of the Bayes classifier assuming that the features obey the multivariate Gaussian distribution. A theoretical analysis that models the number of correctly classified utterances as a hypergeometric random variable enables the derivation of an accurate estimate of the variance of the correct classification rate during cross-validation. By employing such variance estimate, we propose a fast SFFS variant. Experimental findings on Danish emotional speech (DES) and speech under simulated and actual stress (SUSAS) databases demonstrate that SFFS computational time is reduced by 50% and the correct classification rate for classifying speech into emotional states for the selected subset of features varies less than the correct classification rate found by the standard SFFS. Although the proposed SFFS variant is tested in the framework of speech emotion recognition, the theoretical results are valid for any classifier in the context of any wrapper algorithm.

Nonlinear ultrasonic image processing based on signal-adaptive filters and self-organizing neural networks

Two approaches for ultrasonic image processing are examined. First, signal-adaptive maximum likelihood (SAML) filters are proposed for ultrasonic speckle removal. It is shown that in the case of displayed ultrasound (US) image data the maximum likelihood (ML) estimator of the original (noiseless) signal closely resembles the L(2) mean which has been proven earlier to be the ML estimator of the original signal in US B-mode data. Thus, the design of signal-adaptive L(2) mean filters is treated for US B-mode data and displayed US image data as well. Secondly, the segmentation of ultrasonic images using self-organizing neural networks (NN) is investigated. A modification of the learning vector quantizer (L(2 ) LVQ) is proposed in such a way that the weight vectors of the output neurons correspond to the L(2) mean instead of the sample arithmetic mean of the input observations. The convergence in the mean and in the mean square of the proposed L(2) LVQ NN are studied. L(2) LVQ is combined with signal-adaptive filtering in order to allow preservation of image edges and details as well as maximum speckle reduction in homogeneous regions.

Non-Negative Multilinear Principal Component Analysis of Auditory Temporal Modulations for Music Genre Classification

Motivated by psychophysiological investigations on the human auditory system, a bio-inspired two-dimensional auditory representation of music signals is exploited, that captures the slow temporal modulations. Although each recording is represented by a second-order tensor (i.e., a matrix), a third-order tensor is needed to represent a music corpus. Non-negative multilinear principal component analysis (NMPCA) is proposed for the unsupervised dimensionality reduction of the third-order tensors. The NMPCA maximizes the total tensor scatter while preserving the non-negativity of auditory representations. An algorithm for NMPCA is derived by exploiting the structure of the Grassmann manifold. The NMPCA is compared against three multilinear subspace analysis techniques, namely the non-negative tensor factorization, the high-order singular value decomposition, and the multilinear principal component analysis as well as their linear counterparts, i.e., the non-negative matrix factorization, the singular value decomposition, and the principal components analysis in extracting features that are subsequently classified by either support vector machine or nearest neighbor classifiers. Three different sets of experiments conducted on the GTZAN and the ISMIR2004 Genre datasets demonstrate the superiority of NMPCA against the aforementioned subspace analysis techniques in extracting more discriminating features, especially when the training set has small cardinality. The best classification accuracies reported in the paper exceed those obtained by the state-of-the-art music genre classification algorithms applied to both datasets.

Combining text and link analysis for focused crawling-An application for vertical search engines

The number of vertical search engines and portals has rapidly increased over the last years, making the importance of a topic-driven (focused) crawler self-evident. In this paper, we develop a latent semantic indexing classifier that combines link analysis with text content in order to retrieve and index domain-specific web documents. Our implementation presents a different approach to focused crawling and aims to overcome the limitations imposed by the need to provide initial data for training, while maintaining a high recall/precision ratio. We compare its efficiency with other well-known web information retrieval techniques.