Sebastian Schnieder

AVEC 2013: the continuous audio/visual emotion and depression recognition challenge

Mood disorders are inherently related to emotion. In particular, the behaviour of people suffering from mood disorders such as unipolar depression shows a strong temporal correlation with the affective dimensions valence and arousal. In addition, psychologists and psychiatrists take the observation of expressive facial and vocal cues into account while evaluating a patients condition. Depression could result in expressive behaviour such as dampened facial expressions, avoiding eye contact, and using short sentences with flat intonation. It is in this context that we present the third Audio-Visual Emotion recognition Challenge (AVEC 2013). The challenge has two goals logically organised as sub-challenges: the first is to predict the continuous values of the affective dimensions valence and arousal at each moment in time. The second sub-challenge is to predict the value of a single depression indicator for each recording in the dataset. This paper presents the challenge guidelines, the common data used, and the performance of the baseline system on the two tasks.

A review of depression and suicide risk assessment using speech analysis

Review of current diagnostic and assessment methods for depression and suicidality.Review the characteristics of active depressed and suicidal speech databases.Discuss the effects of depression and suicidality on common speech characteristics.Review of studies that use speech to classify or predict depression or suicidality.Discuss future challenges in finding a speech-based markers of either condition. This paper is the first review into the automatic analysis of speech for use as an objective predictor of depression and suicidality. Both conditions are major public health concerns; depression has long been recognised as a prominent cause of disability and burden worldwide, whilst suicide is a misunderstood and complex course of death that strongly impacts the quality of life and mental health of the families and communities left behind. Despite this prevalence the diagnosis of depression and assessment of suicide risk, due to their complex clinical characterisations, are difficult tasks, nominally achieved by the categorical assessment of a set of specific symptoms. However many of the key symptoms of either condition, such as altered mood and motivation, are not physical in nature; therefore assigning a categorical score to them introduces a range of subjective biases to the diagnostic procedure. Due to these difficulties, research into finding a set of biological, physiological and behavioural markers to aid clinical assessment is gaining in popularity. This review starts by building the case for speech to be considered a key objective marker for both conditions; reviewing current diagnostic and assessment methods for depression and suicidality including key non-speech biological, physiological and behavioural markers and highlighting the expected cognitive and physiological changes associated with both conditions which affect speech production. We then review the key characteristics; size, associated clinical scores and collection paradigm, of active depressed and suicidal speech databases. The main focus of this paper is on how common paralinguistic speech characteristics are affected by depression and suicidality and the application of this information in classification and prediction systems. The paper concludes with an in-depth discussion on the key challenges - improving the generalisability through greater research collaboration and increased standardisation of data collection, and the mitigating unwanted sources of variability - that will shape the future research directions of this rapidly growing field of speech processing research.

Applying multiple classifiers and non-linear dynamics features for detecting sleepiness from speech

Comparing different novel feature sets and classifiers for speech processing based fatigue detection is the primary aim of this study. Thus, we conducted a within-subject partial sleep deprivation design (20.00-04.00h, N=77 participants) and recorded 372 speech samples of sustained vowel phonation. The self-report on the Karolinska Sleepiness Scale (KSS) and an observer report on the KSS, the KSS Observer Scale were applied to determine sleepiness reference values. Feature extraction methods of non-linear dynamics (NLD) provide additional information regarding the dynamics and structure of sleepiness speech. In all, 395 NLD features and the 170 phonetic features, which have been computed partially, represent so far unknown auditive-perceptual concepts. Several NLD and phonetic features show significant correlations to KSS ratings, e.g., from the NLD features for male speakers the skewness of vector length within reconstructed phase space (r=.56), and for female speaker the mean of Caos minimum embedding dimensions (r=-.39). After a correlation-filter feature subset selection different classification models and ensemble classifiers (by AdaBoost, Bagging) were trained. Bagging procedures turned out to achieve best performance for male and female speakers on the phonetic and the NLD feature set. The best models for the phonetic feature set achieved 78.3% (NaiveBayes) for male and 68.5% (Bagging Bayes Net) for female speaker classification accuracy in detecting sleepiness. The best model for the NLD feature set achieved 77.2% (Bagging Bayes Net) for male and 76.8% (Bagging Bayes Net) for female speakers. Nevertheless, employing the combined phonetic and NLD feature sets provided additional information and thus resulted in an improved highest UA of 79.6% for male (Bayes Net) and 77.1% for female (AdaBoost Nearest Neighbor) speakers.

Analysis of acoustic space variability in speech affected by depression

Present novel probabilistic acoustic volume, a robust acoustic variability measure.As depression increases phonetic events become concentrated in acoustic space.MFCC feature space becomes tightly concentrated with increasing depression.Speech trajectory in acoustic space becomes smoother with increasing depression.Choice of speech collection paradigm may adversely affect depression detection. The spectral and energy properties of speech have consistently been observed to change with a speakers level of clinical depression. This has resulted in spectral and energy based features being a key component in many speech-based classification and prediction systems. However there has been no in-depth investigation into understanding how acoustic models of spectral features are affected by depression. This paper investigates the hypothesis that the effects of depression in speech manifest as a reduction in the spread of phonetic events in acoustic space as modelled by Gaussian Mixture Models (GMM) in combination with Mel Frequency Cepstral Coefficients (MFCC). Our investigation uses three measures of acoustic variability: Average Weighted Variance (AWV), Acoustic Movement (AM) and Acoustic Volume, which attempt to model depression specific acoustic variations (AWV and Acoustic Volume), or the trajectory of a speech in the acoustic space (AM). Within our analysis we present the Probabilistic Acoustic Volume (PAV) a novel method for robustly estimating Acoustic Volume using a Monte Carlo sampling of the feature distribution being modelled. We show that using an array of PAV points we gain insights into how the concentration of the feature vectors in the feature space changes with depression. Key results - found on two commonly used depression corpora - consistently indicate that as a speakers level of depression increases there are statistically significantly reductions in both AWV (-0.44≤rs≤-0.18 with p<.05) and AM (-0.26≤rs≤-0.19 with p<.05) values, indicating a decrease in localised acoustic variance and smoothing in acoustic trajectory respectively. Further there are also statistically significant reductions (-0.32≤rs≤-0.20 with p<.05) in Acoustic Volume measures and strong statistical evidence (-0.48≤rs≤-0.23 with p<.05) that the MFCC feature space becomes more concentrated. Quantifying these effects is expected to be a key step towards building an objective classification or prediction system which is robust to many of the unwanted - in terms of depression analysis - sources of variability modulated into a speech signal.

Detecting fatigue from steering behaviour applying continuous wavelet transform

The aim of this paper is to develop signal processing based method to measure fatigue from motor behaviour. The advantages of this steering wheel movement approach are that obtaining steering data within driving is robust, non obtrusive, free from sensor application and calibration efforts. Applying methods of continuous wavelet transform (CWT) provides additional information regarding the dynamics and structure of steering behavior comparing to the commonly applied spectral Fourier transform features.

Are men more sleepy than women or does it only look like — Automatic analysis of sleepy speech

The degree of sleepiness in the Sleepy Language Corpus from the Interspeech 2011 Speaker State Challenge is predicted with regression and a very large feature vector. Most notable is the great gender difference which can mainly be attributed to females showing their sleepiness less than males do.

Comparison of in-car touchpads with adaptive haptic feedback

Two in-car touchpads with adaptive haptic feedback are specified in the context of this contribution. These innovative control elements enable an easy and intuitive handling of modern car infotainment systems. The current paper presents the results of a field experiment comparing a touchpad with realistic haptic feedback via sensible and operable elements to a touchpad with haptic feedback via vibration of the touchpad surface in a real driving situation.

Brute Force ECG Feature Extraction Applied on Discomfort Detection

This paper presents the idea of brute force feature extraction for Electrocardiography (ECG) signals applied to discomfort detection. To build an ECG Discomfort Corpus an experimental discomfort induction was conducted. 50 subjects underwent a 2 h (dis-)comfort condition in separate sessions in randomized order. ECG and subjective discomfort was recorded. 5 min ECG segments were labeled with corresponding subjective discomfort ratings, and 6365 brute force features (65 low-level descriptors, first and second order derivatives, and 47 functionals) and 11 traditional heart rate variability (HRV) parameters were extracted. Random Forest machine learning algorithm outperformed SVM and kNN approaches and achieved the best subject-dependent, 10-fold cross-validation results (\(r=.51\)). With this experiment, we are able to show that (a) brute force ECG feature sets achieved better discomfort detection than traditional HRV based ECG feature set; (b) cepstral and spectral flux based features appear to be the most promising to capture HRV phenomena.

EOG feature relevance determination for microsleep detection

Abstract Automatic relevance determination (ARD) was applied to two-channel EOG recordings for microsleep event (MSE) recognition. 10 s immediately before MSE and also before counterexamples of fatigued, but attentive driving were analysed. Two type of signal features were extracted: the maximum cross correlation (MaxCC) and logarithmic power spectral densities (PSD) averaged in spectral bands of 0.5 Hz width ranging between 0 and 8 Hz. Generalised learn-ing vector quantisation (GRLVQ) was used as ARD method to show the potential of feature reduction. This is compared to support-vector machines (SVM), in which the feature reduction plays a much smaller role. Cross validation yielded mean normalised relevancies of PSD features in the range of 1.6 – 4.9 % and 1.9 – 10.4 % for horizontal and vertical EOG, respectively. MaxCC relevancies were 0.002 – 0.006 % and 0.002 – 0.06 %, respectively. This shows that PSD features of vertical EOG are indispensable, whereas MaxCC can be neglected. Mean classification accuracies were estimated at 86.6±b 1.3 % and 92.3±b 0.2 % for GRLVQ and SVM, respectively. GRLVQ permits objective feature reduction by inclusion of all processing stages, but is not as accurate as SVM.

Wege aus der müden (Arbeits-)Gesellschaft: Erklärungsmodelle, Messansätze und Gegenmaßnahmen

ZusammenfassungIn diesem Beitrag werden Ursachenmodelle, Begleiterscheinungen, Messverfahren von Schläfrigkeit und unmittelbare Gegenmaßnahmen ihrer Bewältigung dargestellt. Zunächst werden auf der Grundlage physiologischer und psychologischer Begleitsymptome (automatisierte) Messansätze abgeleitet. Die aufgeführten Bewältigungsstrategien werden auf der Verhaltensebene in stimulierende und deaktivierende eingeteilt, und anschließend um Überlegungen zu verhältnisorientierten erweitert. Abschließend werden die für die individuelle Auswahlentscheidung von Gegenmaßnahmen relevanten situativen und personenbezogenen Determinanten am Beispiel des Napping dargestellt. Ein Rahmenmodell, das die dargestellten relevanten Einflussgrößen mit Blick auf eine effektive Erholungskompetenz beschreibt, wird vorgestellt. Zukünftige Forschungsbemühungen sollten neben der nüchternen ganzheitlichen Analyse von (nicht-)pharmakologischen Stimulanzien insbesondere evaluierte Implementierungen von Napping in den betrieblichen Alltag vornehmen und dabei stärker auf kulturelle Einflussgrößen fokussieren.RésuméDans cette contribution, on a représenté une vue d’ensemble des modèles des causes de la somnolence, ainsi que des phénomènes concomitants, des méthodes de mesures et des contre-mesures immédiates. D’abord, des évaluations de mesures sont dérivées des symptômes d’accompagnement physiologiques et psychologiques. Pour terminer, les stratégies d’accomplissement mentionnées ont été avant tout divisées en stimulant et en désactivant le comportement orienté, avant que des réflexions orientées soient élaborées. Finalement, des contre-mesures dans des situations essentielles et des déterminants relationnels entre personne, à l’exemple du Napping, sont représentés. Un modèle cadre, qui décrit les valeurs d’influence importantes en tenant compte d’une compétence de récupération efficace, est décrit. De futurs efforts de recherche devraient entreprendre en particulier les implémentations évaluées de «napping» dans le quotidien industriel et également se focaliser plus fortement sur les valeurs d’influence culturelle.AbstractSleepiness plays an important role within individual, organizational and economical contexts, especially when focusing on safety, performance or quality of life. Precise measurement of fatigue and sleepiness in professional and private life allows determining the individual’s need for action and serve as integral parts of automated fatigue countermeasure devices. A short overview is given on causes, side effects, measurement instruments and countermeasures of sleepiness.Sleepiness plays an important role within individual, organizational and economical contexts, especially when focusing on safety, performance or quality of life. Precise measurement of fatigue and sleepiness in professional and private life allows determining the individual’s need for action and serve as integral parts of automated fatigue countermeasure devices. A short overview is given on causes, side effects, measurement instruments and countermeasures of sleepiness.