David Sommer

Feature Fusion for the Detection of Microsleep Events

A combination of linear and nonlinear methods for feature fusion is introduced and the performance of this methodology is illustrated on a real-world problem: the detection of sudden and non-anticipated lapses of attention in car drivers due to drowsiness. To achieve this, signals coming from heterogeneous sources are processed, namely the brain electric activity, variation in the pupil size, and eye and eyelid movements. For all the signals considered, the features are extracted both in the spectral domain and in state space. Linear features are obtained by the modified periodogram, whereas the nonlinear features are based on the recently introduced method of delay vector variance (DVV). The decision process based on such fused features is achieved by support vector machines (SVM) and learning vector quantization (LVQ) neural networks. For the latter also methods of metrics adaptation in the input space are applied. The parameters of all utilized algorithms are optimized empirically in order to gain maximal classification accuracy. It is also shown that metrics adaptation by weighting the input features can improve the classification accuracy, but only to a limited extent. Limited improvements are also obtained when fusing features of selected signals, but highest improvements are gained by fusion of features of all available signals. In this case test errors are reduced down to 9% in the mean, which clearly illustrates the potential of our methodology to establish a reference standard of drowsiness and microsleep detection devices for future online driver monitoring.

Fusion of state space and frequency-domain features for improved microsleep detection

A novel approach for Microsleep Event detection is presented. This is achieved based on multisensor electroencephalogram (EEG) and electrooculogram (EOG) measurements recorded during an overnight driving simulation task. First, using video clips of the driving, clear Microsleep (MSE) and Non-Microsleep (NMSE) events were identified. Next, segments of EEG and EOG of the selected events were analyzed and features were extracted using Power Spectral Density and Delay Vector Variance. The so obtained features are used in several combinations for MSE detection and classification by means of populations of Learning Vector Quantization (LVQ) networks. Best classification results, with test errors down to 13%, were obtained by a combination of all the recorded EEG and EOG channels, all features, and with feature relevance adaptation using Genetic Algorithms.

Blind Extraction of Microsleep Events

The aim of this study is to detect the occurrence of microsleep events in an overnight driving task. We propose a biosignal analysis method for the detection and extraction of microsleep events. This is achieved by employing blind source extraction method based on a cascaded nonlinear estimator to extract the relevant microsleep events. The cascaded nonlinear estimator jointly estimates the kurtosis and measure the nonlinearity and noise effects within the biosignal. This proposed method was applied to the electroencephalogram and electroculogram recorded of 12 young volunteers while performing monotonic overnight driving in a real car driving simulation laboratory. The extracted microsleep signals can then be used for driving simulation pilot studies for alertness monitoring, and to trigger the activation of alertness countermeasures.

The performance of LVQ based automatic relevance determination applied to spontaneous biosignals

The issue of Automatic Relevance Determination (ARD) has attracted attention over the last decade for the sake of efficiency and accuracy of classifiers, and also to extract knowledge from discriminant functions adapted to a given data set. Based on Learning Vector Quantization (LVQ), we recently proposed an approach to ARD utilizing genetic algorithms. Another approach is the Generalized Relevance LVQ which has been shown to outperform other algorithms of the LVQ family. In the following we present a unique description of a number of LVQ algorithms and compare them concerning their classification accuracy and their efficacy. For this purpose a real world data set consisting of spontaneous EEG and EOG during overnight-driving is employed to detect so-called microsleep events. Results show that relevance learning can improve classification accuracies, but do not reach the performance of Support Vector Machines. The computational costs for the best performing classifiers are exceptionally high and exceed basic LVQ1 by a factor of 104.

Detecting fatigue from steering behaviour applying continuous wavelet transform

The aim of this paper is to develop signal processing based method to measure fatigue from motor behaviour. The advantages of this steering wheel movement approach are that obtaining steering data within driving is robust, non obtrusive, free from sensor application and calibration efforts. Applying methods of continuous wavelet transform (CWT) provides additional information regarding the dynamics and structure of steering behavior comparing to the commonly applied spectral Fourier transform features.

Assessing Driver’s Hypovigilance from Biosignals

For the assessment of Fatigue Monitoring Technologies (FMT) an independent reference of driver’s hypovigilance is needed. To achieve this goal, we propose to process EEG and EOG biosignals, to apply a feature fusion concept and to utilize Support-Vector Machines (SVM) for classification. Karolinska Sleepiness Scale (KSS) and variation of lane deviation (VLD) were used in order to get independent class labels, whereas KSS are subjective and VLD are objective measures. For simplicity, two classes were determined: slight and strong hypovigilance. 16 young volunteers participated in overnight experiments in our real car driving simulation lab. Results were compared with PERCLOS (percentage of eye closure), an oculomotoric variable utilized in several FMT systems. We conclude that EEG and EOG biosignals contain substantial higher amount of hypovigilance information than the PERCLOS biosignal.

Automatic Knowledge Extraction: Fusion of Human Expert Ratings and Biosignal Features for Fatigue Monitoring Applications

A framework for automatic relevance determination based on artificial neural networks and evolution strategy is presented. It is applied for an important problem in biomedicine, namely the detection of unintentional episodes of sleep during sustained operations of subjects, so-called microsleep episodes. Human expert ratings based on video and biosignal recordings are necessary to judge microsleep episodes. Ratings are fused together with linear and nonlinear features which are extracted from three types of biosignals: electroencephalography, electrooculography, and eyetracking. Changes in signal modality due to nonlinearity and stochasticity are quantified by the ‘delay vector variance’ method. Results show large inter-individual variability. Though the framework is outperformed by support vector machines in terms of classification accuracy, the estimated relevance values provide knowledge of signal characteristics during microsleep episodes.

Applying nonlinear dynamics features for speech-based fatigue detection

This paper describes a speech signal processing method to measure fatigue from speech. The advantages of this realtime approach are that obtaining speech data is non obtrusive, free from sensor application and calibration efforts. Applying methods of Non Linear Dynamics(NLD) provides additional information regarding the dynamics and structure of fatigue speech comparing to the commonly applied speech emotion recognition feature set (e.g. fundamental frequency, intensity, pause patterns, formants, cepstral coefficients). We achieved significant correlations between fatigue and NLD features of 0.29. The validity of this approach is briefly discussed by summarizing the empirical results of a sleep deprivation study.

Biosignal Based Discrimination between Slight and Strong Driver Hypovigilance by Support-Vector Machines

In the area of transportation research, there is a growing need for robust and reliable measures of hypovigilance, particularly due to the current volume of research in the development and validation of Fatigue Monitoring Technologies (FMT). Most of the currently emerging FMT is vision based. The parameter Percentage of Eyelid Closure (PERCLOS) is used for the fatigue detection. The development and validation of PERCLOS based FMT require an independent reference standard of drivers’ hypovigilance. Most approaches utilized electrooculography (EOG) and electroencephalography (EEG) combined with descriptive statistics of a few time or spectral domain features. Typically, the power spectral densities (PSD) averaged in four to six spectral bands is used for fatigue characterization. This constricted approach led to sometimes contradicting results and questioned the validity of the EEG and EOG as gold standard for driver fatigue, wrongly as we will show. Here we present a more general approach using generalized EEG and EOG PSD features in combination with data fusion and advanced computational intelligence methods, such as Support-Vector Machines (SVM). Biosignal based discrimination of driver hypovigilance was performed by independent class labels which were derived from Karolinska Sleepiness Scale (KSS) and from variation of lane deviation (VLD). The first is a measure of subjectively self-experienced hypovigilance, whereas the second is an objective measure of performance decrements. For simplicity, two label classes were discriminated: slight and strong hypovigilance. The discrimination results of PERCLOS were compared with results from single and combined EEG and EOG channels. We conclude that EEG and EOG biosignals are substantially more suited to assess driver’s hypovigilance than the PERCLOS biosignals. In addition, computational intelligence performed better when objective class labels were used instead of subjective class labels.

Alertness assessment using data fusion and discrimination ability of LVQ-networks

To track the alertness changes of 14 subjects during a night driving simulation study traditional alertness measures such Visual Analog Sleepiness Scale, Alpha Attenuation Test (AAT), and number of Microsleep events per driving session were used. The aim of the paper is to assess these traditional alertness measures regarding their mutual correlations, revise one of them (AAT) and introduce new more general methods to capture changes in human alertness without too many constraints attached. The applied methods are utilizing data fusion methods and data discrimination capabilities via Learning Vector Quantification networks. The advantage of using more general data analysis methods which allows one to assess the validity of proposed alertness measures and opens possibilities to get a more comprehensive knowledge of obtained results.