Walid Karam

Talking-face identity verification, audiovisual forgery, and robustness issues

The robustness of a biometric identity verification (IV) system is best evaluated by monitoring its behavior under impostor attacks. Such attacks may include the transformation of one, many, or all of the biometric modalities. In this paper, we present the transformation of both speech and visual appearance of a speaker and evaluate its effects on the IV system. We propose MixTrans, a novel method for voice transformation. MixTrans is a mixture-structured bias voice transformation technique in the cepstral domain, which allows a transformed audio signal to be estimated and reconstructed in the temporal domain. We also propose a face transformation technique that allows a frontal face image of a client speaker to be animated. This technique employs principal warps to deform defined MPEG-4 facial feature points based on determined facial animation parameters (FAPs). The robustness of the IV system is evaluated under these attacks.

Multimodal Human Machine Interactions in Virtual and Augmented Reality

Virtual worlds are developing rapidly over the Internet. They are visited by avatars and staffed with Embodied Conversational Agents (ECAs). An avatar is a representation of a physical person. Each person controls one or several avatars and usually receives feedback from the virtual world on an audio-visual display. Ideally, all senses should be used to feel fully embedded in a virtual world. Sound, vision and sometimes touch are the available modalities. This paper reviews the technological developments which enable audio-visual interactions in virtual and augmented reality worlds. Emphasis is placed on speech and gesture interfaces, including talking face analysis and synthesis.

An improved real-time method for counting people in crowded scenes based on a statistical approach

In this paper, we present a real-time method for counting people in crowded conditions using an indirect/statistical approach. Our method is based on an algorithm by Albiol et al. that won the PETS 2009 contest on people counting. We employ a scale-invariant interest point detector from the state of the art coined SURF (Speeded-Up Robust Features), and we exploit motion information to retain only interest points belonging to moving people. Direct proportionality is then assumed between the number of remaining SURF points and the number of people. Our technique was first tested on three video sequences from the PETS dataset. Results showed an improvement over Albiols in all the three cases. It was then tested on our set of video sequences taken under various conditions. Despite the complexity of the scenes, results were very reasonable with a mean relative error ranging from 9.36% to 17.06% and a mean absolute error ranging from 1.13 to 3.33. Testing this method on a new dataset proved its speed and accuracy under many shooting scenarios, especially in crowded conditions where the averaging process reduces the variations in the number of detected SURF points per person.

An audio-visual imposture scenario by talking face animation

We describe a system that allows an impostor to lead an audio-visual telephone conversation, and sign data electronically on behalf of an authorized client. During the conversation, audio and video of the impostor are altered so as to mimic the client. The voice of an impostor is processed and used to reproduce the voice of the authorized client. Speech segments obtained from clients recordings are used to synthesize new sentences that the client never pronounced. On the visual side, the imposters talking face is detected and facial features are extracted and used to animate a synthetic talking face. The texture of the impersonated face is mapped onto the talking head and coded for transmission over the phone, along with the synthesized voice. Audio-visual coding and synthesis is realized by indexing in a memory containing audio-visual sequences. Stochastic models (coupled HMM) of characteristic segments are used to drive the memory search.

Identities, forgeries and disguises

The preservation of your identity could become a major concern. In many situations, you need to claim an identity and this claim needs to be verified somehow. The technology called biometrics may help. But, what if a deliberate impostor claims your identity? Will this forgery be always detected? Biometric identity verification is imperfect. This paper reviews some of the techniques that a deliberate impostor could use to defeat a biometric verification system. It focuses on audio-visual forgeries using voice conversion and face animation. It also describes identity disguise as a means of falsifying and concealing one|s identity. The recovery of an identity and cancellable biometrics are also useful techniques to protect from identity theft. Such techniques could also find useful applications in multimedia.

Audio-Visual Identity Verification and Robustness to Imposture

The robustness of talking-face identity verification (IV) systems is best evaluated by monitoring their behavior under impostor attacks. We propose a scenario where the impostor uses a still face picture and a sample of speech of the genuine client to transform his/her speech and visual appearance into that of the target client. We propose MixTrans , an original text-independent technique for voice transformation in the cepstral domain, which allows a transformed audio signal to be estimated and reconstructed in the temporal domain. We also propose a face transformation technique that allows a frontal face image of a client to be animated, using principal warps to deform defined MPEG-4 facial feature points based on determined facial animation parameters. The robustness of the talking-face IV system is evaluated under these attacks. Results on the BANCA talking-face database clearly show that such attacks represent a serious challenge and a security threat to IV systems.

Biometrics security and experiments on face recognition algorithms

Biometrics security analysis and performance evaluation of the following Face Recognition Algorithms is performed: Principal Components Analysis (PCA), Linear Discriminant Analysis (LDA) and Bayesian Intrapersonal/Extrapersonal Classifier (BIC), using the BANCA database. Software tools retrieve and preprocess images from sequential records within the BANCA database for algorithm evaluation. Then a verification environment over the set of images to be tested is developed, the above algorithms are invoked over the verification set, and verification parameters are collected. Results proved PCA performed most accurately and effectively with regards to security concerns, with an average recognition rate of 93%, while LDA and BIC lagged behind with recognition rates ranging from 80%-83%.

Identity Verification robustness to audiovisual imposture

The robustness of a biométrie Identity Verification (IV) system is best evaluated by monitoring its behavior under impostor attacks. Such attacks may include the transformation of one or multiple biométrie modalities typically face and voice. In this paper, we evaluate the effects of transforming both the face and the voice of a speaker on an IV system. Results of imposture experiments on the BANCA [1] talking-face database show an increase in the error rate of the audiovisual IV system, which indicates a higher acceptance rate of the claimed identities. Results are reported on transformation of the speech and visual signals (MixTrans for speech, and TPS warping animation for face) [2] For both types of forgeries, the verification system accepted more impostors.

Audio-visual imposture

A GMM based audio visual speaker verification system is described and an Active Appearance Model with a linear speaker transformation system is used to evaluate the robustness of the verification. An Active Appearance Model (AAM) is used to automatically locate and track a speakers face in a video recording. A Gaussian Mixture Model (GMM) based classifier (BECARS) is used for face verification. GMM training and testing is accomplished on DCT based extracted features of the detected faces. On the audio side, speech features are extracted and used for speaker verification with the GMM based classifier. Fusion of both audio and video modalities for audio visual speaker verification is compared with face verification and speaker verification systems. To improve the robustness of the multimodal biometric identity verification system, an audio visual imposture system is envisioned. It consists of an automatic voice transformation technique that an impostor may use to assume the identity of an authorized client. Features of the transformed voice are then combined with the corresponding appearance features and fed into the GMM based system BECARS for training. An attempt is made to increase the acceptance rate of the impostor and to analyzing the robustness of the verification system. Experiments are being conducted on the BANCA database, with a prospect of experimenting on the newly developed PDAtabase developed within the scope of the SecurePhone project.