Rubén Fernández Pozo

Usability evaluation of multi-modal biometric verification systems

As a result of the evolution in the field of biometrics, a new breed of techniques and methods for user identity recognition and verification has appeared based on the recognition and verification of several biometric features considered unique to each individual. Signature and voice characteristics, facial features, and iris and fingerprint patterns have all been used to identify a person or just to verify that the person is who he/she claims to be. Although still relatively new, these new technologies have already reached a level of development that allows its commercialization. However, there is a lack of studies devoted to the evaluation of these technologies from a user-centered perspective. This paper is intended to promote user-centered design and evaluation of biometric technologies. Towards this end, we have developed a platform to perform empirical evaluations of commercial biometric identity verification systems, including fingerprint, voice and signature verification. In this article, we present an initial empirical study in which we evaluate, compare and try to get insights into the factors that are crucial for the usability of these systems.

Assessment of severe apnoea through voice analysis, automatic speech, and speaker recognition techniques

This study is part of an ongoing collaborative effort between the medical and the signal processing communities to promote research on applying standard Automatic Speech Recognition (ASR) techniques for the automatic diagnosis of patients with severe obstructive sleep apnoea (OSA). Early detection of severe apnoea cases is important so that patients can receive early treatment. Effective ASR-based detection could dramatically cut medical testing time. Working with a carefully designed speech database of healthy and apnoea subjects, we describe an acoustic search for distinctive apnoea voice characteristics. We also study abnormal nasalization in OSA patients by modelling vowels in nasal and nonnasal phonetic contexts using Gaussian Mixture Model (GMM) pattern recognition on speech spectra. Finally, we present experimental findings regarding the discriminative power of GMMs applied to severe apnoea detection. We have achieved an 81% correct classification rate, which is very promising and underpins the interest in this line of inquiry.

Analysis of voice features related to obstructive sleep apnoea and their application in diagnosis support

Obstructive sleep apnoea (OSA) is a highly prevalent disease affecting an estimated 2-4% of the adult male population that is difficult and very costly to diagnose because symptoms can remain unnoticed for years. The reference diagnostic method, Polysomnography (PSG), requires the patient to spend a night at the hospital monitored by specialized equipment. Therefore fast and less costly screening techniques are normally applied for setting priorities to proceed to the polysomnography diagnosis. In this article the use of speech analysis is proposed as an alternative or complement to existing screening methods. A set of voice features that could be related to apnoea are defined, based on previous results from other authors and our own analysis. These features are analyzed first in isolation and then in combination to assess their discriminative power to classify voices as corresponding to apnoea patients and healthy subjects. This analysis is performed in a database containing three repetitions of four carefully designed sentences read by 40 healthy subjects and 42 subjects suffering from severe apnoea. As a result of the analysis, a linear discriminant model (LDA) was defined including a subset of eight features (signal-to-disperiodicity ratio, a nasality measure, harmonic-to-noise ratio, jitter, difference between third and second formants on a specific vowel, duration of two of the sentences and the percentage of silence in one of the sentences). This model was tested on a separate database containing 20 healthy and 20 apnoea subjects yielding a sensitivity of 85% and a specificity of 75%, with a F1-measure of 81%. These results indicate that the proposed method, only requiring a few minutes to record and analyze the patients voice during the visit to the specialist, could help in the development of a non-intrusive, fast and convenient PSG-complementary screening technique for OSA.

Formant Frequencies and Bandwidths in Relation to Clinical Variables in an Obstructive Sleep Apnea Population.

OBJECTIVES We investigated whether differences in formants and their bandwidths, previously reported comparing small sample population of healthy individuals and patients with obstructive sleep apnea (OSA), are detected on a larger population representative of a clinical practice scenario. We examine possible indirect or mediated effects of clinical variables, which may shed some light on the connection between speech and OSA. STUDY DESIGN In a retrospective study, 241 male subjects suspected to suffer from OSA were examined. The apnea-hypopnea index (AHI) was obtained for every subject using overnight polysomnography. Furthermore, the clinical variables usually reported as predictors of OSA, body mass index (BMI), cervical perimeter, height, weight, and age, were collected. Voice samples of sustained phonations of the vowels /a/, /e/, /i/, /o/, and /u/ were recorded. METHODS Formant frequencies F1, F2, and F3 and bandwidths BW1, BW2, and BW3 of the sustained vowels were determined using spectrographic analysis. Correlations among AHI, clinical parameters, and formants and bandwidths were determined. RESULTS Correlations between AHI and clinical variables were stronger than those between AHI and voice features. AHI only correlates poorly with BW2 of /a/ and BW3 of /e/. A number of further weak but significant correlations have been detected between voice and clinical variables. Most of them were for height and age, with two higher values for age and F2 of /o/ and F2 of /u/. Only few very weak correlations were detected between voice and BMI, weight and cervical perimeter, wich are the clinical variables more correlated with AHI. CONCLUSIONS No significant correlations were detected between AHI and formant frequencies and bandwidths. Correlations between voice and other clinical factors characterizing OSA are weak but highlight the importance of considering indirect or mediated effects of such clinical variables in any research on speech and OSA.

Using HMM to Detect Speakers with Severe Obstructive Sleep Apnoea Syndrome

Nowadays definitive diagnosis of obstructive sleep apnoea (OSA) syndrome is expensive and time-consuming. Previous research on voice characteristics of OSA patients has shown that resonance, phonation and articulation differences arise when compared to healthy subjects. In this contribution we study different speech modeling techniques to detect patients with severe OSA envisioning the future classification of patients according to their priority of need identifying the most severe cases and reducing medical costs.

Estimating Vehicle Movement Direction from Smartphone Accelerometers Using Deep Neural Networks

Characterization of driving maneuvers or driving styles through motion sensors has become a field of great interest. Before now, this characterization used to be carried out with signals coming from extra equipment installed inside the vehicle, such as On-Board Diagnostic (OBD) devices or sensors in pedals. Nowadays, with the evolution and scope of smartphones, these have become the devices for recording mobile signals in many driving characterization applications. Normally multiple available sensors are used, such as accelerometers, gyroscopes, magnetometers or the Global Positioning System (GPS). However, using sensors such as GPS increase significantly battery consumption and, additionally, many current phones do not include gyroscopes. Therefore, we propose the characterization of driving style through only the use of smartphone accelerometers. We propose a deep neural network (DNN) architecture that combines convolutional and recurrent networks to estimate the vehicle movement direction (VMD), which is the forward movement directional vector captured in a phone’s coordinates. Once VMD is obtained, multiple applications such as characterizing driving styles or detecting dangerous events can be developed. In the development of the proposed DNN architecture, two different methods are compared. The first one is based on the detection and classification of significant acceleration driving forces, while the second one relies on longitudinal and transversal signals derived from the raw accelerometers. The final success rate of VMD estimation for the best method is of 90.07%.Characterization of driving maneuvers or driving styles through motion sensors has become a field of great interest. Before now, this characterization used to be carried out with signals coming from extra equipment installed inside the vehicle, such as On-Board Diagnostic (OBD) devices or sensors in pedals. Nowadays, with the evolution and scope of smartphones, these have become the devices for recording mobile signals in many driving characterization applications. Normally multiple available sensors are used, such as accelerometers, gyroscopes, magnetometers or the Global Positioning System (GPS). However, using sensors such as GPS increase significantly battery consumption and, additionally, many current phones do not include gyroscopes. Therefore, we propose the characterization of driving style through only the use of smartphone accelerometers. We propose a deep neural network (DNN) architecture that combines convolutional and recurrent networks to estimate the vehicle movement direction (VMD), which is the forward movement directional vector captured in a phones coordinates. Once VMD is obtained, multiple applications such as characterizing driving styles or detecting dangerous events can be developed. In the development of the proposed DNN architecture, two different methods are compared. The first one is based on the detection and classification of significant acceleration driving forces, while the second one relies on longitudinal and transversal signals derived from the raw accelerometers. The final success rate of VMD estimation for the best method is of 90.07%.

Beyond objective performance evaluation in multimodal biometric systems

Biometric identity verification is a reality today. However, this relatively new field still requires a large amount of user-centred studies before becoming commonly used.This paper presents a user-centred analysis of a multimodal authentication system for secure Internet access where users can choose freely between three different biometric modalities (fingerprint, voice and signature) to enrol, verify their identity and act as impostors in an unsupervised manner, aided only by an automated embodied conversational agent.Objective and subjective information was collected to analyse relevant relationships between authentication performance, ergonomie issues and user preconceptions and impressions. Particular attention has been paid to analyse also the evolution of users’ choices of modality. From the results of our study we infer usability insights for the design of multimodal biometric security systems, and point towards directions of further inquiry.RésuméAujourd’hui l’authentification de l’identité grâce à la biométrique est une réalité. Cependant, cette technologie, relativement récente, requiert une grande quantitée d’études centrées sur l’utilisateur avant de pouvoir être communément utilisée.Cet article, basé sur une étude de l’utilisateur, présente l’analyse d’un système multimodal d’authentification pour l’accès sécurisé à l’Internet, où les utilisateurs peuvent choisir librement parmis trois modalités biométriques différentes (l’empreinte digitale, la voix et la signature) pour s’inscrire, vérifier leurs identités et agir en tant qu ’imposteurs, sans personnel d’encadrement mais avec l’aide d’un agent conversationnel animé.Des informations objectives et subjectives ont été recueillies afin d’analyser les relations entre les performances d’authentification, l’ergonomie et les avis de l’utilisateur. Nous avons analysé aussi l’évolution des choix de modalité des utilisateurs. Nous déduirons des idées pour la conception des systèmes biométriques multimodaux de sécurité, et l’orientation d’études plus approfondies.