Udo Trutschel

Feature Fusion for the Detection of Microsleep Events

A combination of linear and nonlinear methods for feature fusion is introduced and the performance of this methodology is illustrated on a real-world problem: the detection of sudden and non-anticipated lapses of attention in car drivers due to drowsiness. To achieve this, signals coming from heterogeneous sources are processed, namely the brain electric activity, variation in the pupil size, and eye and eyelid movements. For all the signals considered, the features are extracted both in the spectral domain and in state space. Linear features are obtained by the modified periodogram, whereas the nonlinear features are based on the recently introduced method of delay vector variance (DVV). The decision process based on such fused features is achieved by support vector machines (SVM) and learning vector quantization (LVQ) neural networks. For the latter also methods of metrics adaptation in the input space are applied. The parameters of all utilized algorithms are optimized empirically in order to gain maximal classification accuracy. It is also shown that metrics adaptation by weighting the input features can improve the classification accuracy, but only to a limited extent. Limited improvements are also obtained when fusing features of selected signals, but highest improvements are gained by fusion of features of all available signals. In this case test errors are reduced down to 9% in the mean, which clearly illustrates the potential of our methodology to establish a reference standard of drowsiness and microsleep detection devices for future online driver monitoring.

Data fusion for modern engineering applications: an overview

An overview of data fusion approaches is provided from the signal processing viewpoint. The general concept of data fusion is introduced, together with the related architectures, algorithms and performance aspects. Benefits of such an approach are highlighted and potential applications are identified. Case studies illustrate the merits of applying data fusion concepts in real world applications.

Fusion of state space and frequency-domain features for improved microsleep detection

A novel approach for Microsleep Event detection is presented. This is achieved based on multisensor electroencephalogram (EEG) and electrooculogram (EOG) measurements recorded during an overnight driving simulation task. First, using video clips of the driving, clear Microsleep (MSE) and Non-Microsleep (NMSE) events were identified. Next, segments of EEG and EOG of the selected events were analyzed and features were extracted using Power Spectral Density and Delay Vector Variance. The so obtained features are used in several combinations for MSE detection and classification by means of populations of Learning Vector Quantization (LVQ) networks. Best classification results, with test errors down to 13%, were obtained by a combination of all the recorded EEG and EOG channels, all features, and with feature relevance adaptation using Genetic Algorithms.

Experimental verification of a virtual-mode treatment for the excitation of surface plasmon polaritons by attenuated total reflection

Attenuated-total-reflection spectra obtained by the optical excitation of surface plasmon polaritons can be interpreted in terms of two different virtual modes. It is shown that the HWHM of absorptance as a function of the incidence angle can be described by the decay constant of one of the modes. The attenuated-total-reflection resonance angle, on the other hand, is determined by the phase of another virtual mode. An experimental verification of this virtual-mode treatment is carried out for thick aluminum films.

Assessing Driver’s Hypovigilance from Biosignals

For the assessment of Fatigue Monitoring Technologies (FMT) an independent reference of driver’s hypovigilance is needed. To achieve this goal, we propose to process EEG and EOG biosignals, to apply a feature fusion concept and to utilize Support-Vector Machines (SVM) for classification. Karolinska Sleepiness Scale (KSS) and variation of lane deviation (VLD) were used in order to get independent class labels, whereas KSS are subjective and VLD are objective measures. For simplicity, two classes were determined: slight and strong hypovigilance. 16 young volunteers participated in overnight experiments in our real car driving simulation lab. Results were compared with PERCLOS (percentage of eye closure), an oculomotoric variable utilized in several FMT systems. We conclude that EEG and EOG biosignals contain substantial higher amount of hypovigilance information than the PERCLOS biosignal.

Biosignal Based Discrimination between Slight and Strong Driver Hypovigilance by Support-Vector Machines

In the area of transportation research, there is a growing need for robust and reliable measures of hypovigilance, particularly due to the current volume of research in the development and validation of Fatigue Monitoring Technologies (FMT). Most of the currently emerging FMT is vision based. The parameter Percentage of Eyelid Closure (PERCLOS) is used for the fatigue detection. The development and validation of PERCLOS based FMT require an independent reference standard of drivers’ hypovigilance. Most approaches utilized electrooculography (EOG) and electroencephalography (EEG) combined with descriptive statistics of a few time or spectral domain features. Typically, the power spectral densities (PSD) averaged in four to six spectral bands is used for fatigue characterization. This constricted approach led to sometimes contradicting results and questioned the validity of the EEG and EOG as gold standard for driver fatigue, wrongly as we will show. Here we present a more general approach using generalized EEG and EOG PSD features in combination with data fusion and advanced computational intelligence methods, such as Support-Vector Machines (SVM). Biosignal based discrimination of driver hypovigilance was performed by independent class labels which were derived from Karolinska Sleepiness Scale (KSS) and from variation of lane deviation (VLD). The first is a measure of subjectively self-experienced hypovigilance, whereas the second is an objective measure of performance decrements. For simplicity, two label classes were discriminated: slight and strong hypovigilance. The discrimination results of PERCLOS were compared with results from single and combined EEG and EOG channels. We conclude that EEG and EOG biosignals are substantially more suited to assess driver’s hypovigilance than the PERCLOS biosignals. In addition, computational intelligence performed better when objective class labels were used instead of subjective class labels.

Alertness assessment using data fusion and discrimination ability of LVQ-networks

To track the alertness changes of 14 subjects during a night driving simulation study traditional alertness measures such Visual Analog Sleepiness Scale, Alpha Attenuation Test (AAT), and number of Microsleep events per driving session were used. The aim of the paper is to assess these traditional alertness measures regarding their mutual correlations, revise one of them (AAT) and introduce new more general methods to capture changes in human alertness without too many constraints attached. The applied methods are utilizing data fusion methods and data discrimination capabilities via Learning Vector Quantification networks. The advantage of using more general data analysis methods which allows one to assess the validity of proposed alertness measures and opens possibilities to get a more comprehensive knowledge of obtained results.

Heart rate measures reflect the interaction of low mental workload and fatigue during driving simulation

The objective of this study was to assess the monotonic mental workload under changing conditions of operator fatigue during a night time driver simulation study. Several cardiovascular measures were used in order to differentiate between driving and a continuous tracking task. From all of the standard cardiovascular measures, heart rate in beats per minute emerged as the most sensitive for workload discrimination. Heart rate was higher during driving than during the tracking task, pointing to a slightly higher demanding workload for the driving task. This result was stable over the course of the night and showed only a minimal fatigue influence. Heart rate variability in milliseconds, on the other hand, was on average higher for the continuous tracking task in comparison to the driving. This was especially the case for the sessions with high subjective sleepiness. It can thus be concluded that the fatigue state of the operator was more impaired during the tracking task than during driving.

Discrimination and relevance determination of heart rate variability features for the identification of congestive heart failure

We propose a machine learning framework that implements automated relevance determination in order to identify the deciding RR interval features for the discrimination between congestive heart failure and healthy condition. As a result, the most relevant features of heart rate variability (HRV) are narrowly located spectral components in the very-low and low frequency band, and specific ordinal patterns. HRV is generally reduced in comparison to the healthy condition; also the autonomic regulation of heart rate acceleration and deceleration appears to be pathlogically inversed.

Discrimination power of spectral and nonlinear heart rate variability features for the identification of congestive heart failure

Recognizing pathological heart rhythm features remains a challenge of cardiovascular research. We adopt a machine learning framework with empirically optimized parameters to distinguish heart failure from healthy condition, emphasizing on spectral and nonlinear features of heart rate variability. Fine-grained spectral power densities of RR intervals emerged as the best discriminating group of features, yielding a classification error rate of 13.6 % when presented at a segment length of 50 minutes.