Jukka Holappa

An Optimized Illumination Normalization Method for Face Recognition

Differences in illumination conditions cause significant challenges for any 2-D face recognition algorithm. One of the methods to counter these effects is image preprocessing before feature extraction. In this paper we present a new preprocessing approach that uses custom filters obtained through an optimization procedure striving for most suitable preprocessing filters for the selected feature extractor and distance measure. We experiment with it using Local Binary Pattern texture features and X2 histogram distance metric. Results are provided for Face Recognition Grand Challenge (FRGC) 1.0.4 dataset.

Minotaurus: A System for Affective Human–Robot Interaction in Smart Environments

A service robot should be able to detect and identify the user in order to personalize its services and guarantee security, it should recognize the user’s emotions to allow affective interaction, and it should be able to communicate easily with the user and understand given commands by recognizing speech and gestures. Our research is motivated, for example, by the emerging needs of elderly care, health care, safety, and logistics. We have developed a distributed system for affective human–robot interaction which combines all the basic sensory elements and can collaborate with a smart environment and obtain further knowledge from the Internet. The realized HRI robot, Minotaurus, runs in real time and is capable of interacting with human beings. Minotaurus forms a rather generic platform for experimenting with human–robot collaboration in different applications and environments.

MORE --- a multimodal observation and analysis system for social interaction research

The MORE system is designed for observation and machine-aided analysis of social interaction in real life situations, such as classroom teaching scenarios and business meetings. The system utilizes a multichannel approach to collect data whereby multiple streams of data in a number of different modalities are obtained from each situation. Typically the system collects a 360-degree video and audio feed from multiple microphones set up in the space. The system includes an advanced server backend component that is capable of performing video processing, feature extraction and archiving operations on behalf of the user. The feature extraction services form a key part of the system and rely on advanced signal analysis techniques, such as speech processing, motion activity detection and facial expression recognition in order to speed up the analysis of large data sets. The provided web interface weaves the multiple streams of information together, utilizes the extracted features as metadata on the audio and video data and lets the user dive into analyzing the recorded events. The objective of the system is to facilitate easy navigation of multimodal data and enable the analysis of the recorded situations for the purposes of, for example, behavioral studies, teacher training and business development. A further unique feature of the system is its low setup overhead and high portability as the lightest MORE setup only requires a laptop computer and the selected set of sensors on site.

OMEG: Oulu Multi-pose Eye Gaze Dataset

Data is in a very important position for pattern recognition tasks including eye gaze estimation. In the literature, most researchers used normal face datasets, which are not specifically designed for eye gaze estimation. As a result, it is difficult to obtain fine labeled eye gaze direction. Therefore large datasets with well-defined gaze directions are desired.

Fast and accurate multi-view reconstruction by multi-stage prioritised matching

In this study, the authors propose a multi-view stereo reconstruction method which creates a three-dimensional point cloud of a scene from multiple calibrated images captured from different viewpoints. The method is based on a prioritised match expansion technique, which starts from a sparse set of seed points, and iteratively expands them into neighbouring areas by using multiple expansion stages. Each seed point represents a surface patch and has a position and a surface normal vector. The location and surface normal of the seeds are optimised using a homography-based local image alignment. The propagation of seeds is performed in a prioritised order in which the most promising seeds are expanded first and removed from the list of seeds. The first expansion stage proceeds until the list of seeds is empty. In the following expansion stages, the current reconstruction may be further expanded by finding new seeds near the boundaries of the current reconstruction. The prioritised expansion strategy allows efficient generation of accurate point clouds and their experiments show its benefits compared with non-prioritised expansion. In addition, a comparison to the widely used patch-based multi-view stereo software shows that their method is significantly faster and produces more accurate and complete reconstructions.

On the robustness of audiovisual liveness detection to visual speech animation

Audiovisual speech synchrony detection is an important liveness check for talking face verification systems to make sure that the (pre-defined) content and timing of the given audible and visual speech samples match. Nowadays, there exists virtually no technical limitations for combining transferable facial animation and voice conversion (or synthesis) to create an ultimate audiovisual artifact that is able to spoof even advanced random challenge-response based liveness detection. In this study, we investigate the performance of the state-of-the-art text-independent lip-sync detection techniques under presentation attacks consisting of audio recordings of the targeted person and corresponding animated visual speech. Our experimental analysis with three different photo-realistic visual speech animation techniques reveals that generic synchrony models can be fooled even with underarticulated but synchronized lip movements. Thus, measuring audio-video synchrony or content alone is not enough for securing audiovisual biometric systems. Our preliminary findings suggest though that adaptation of person-specific audiovisual speech dynamics is one possible approach to tackle these kinds of high-effort attacks.

Data-driven techniques for smoothing histograms of local binary patterns

Local binary pattern histograms have proved very successful texture descriptors. Despite this success, the description procedure bears some drawbacks that are still lacking solutions in the literature. One of the problems arises when the number of extractable local patterns reduces while their dimension increases rendering the output histogram descriptions sparse and unstable, finally showing up as a reduced recognition rate. A smoothing method based on kernel density estimation was recently proposed as a means to tackle the aforementioned problem. A constituent part of the method is to determine how much to smooth a histogram. Previously, this was solved via trial-and-error in a problem-specific manner. In this paper, the goal is to present data-driven methods to determine this smoothing automatically. In the end, we present unsupervised and supervised methods for the given task and validate their performance with a representative set of local binary pattern variants in texture analysis problems covering material categorization and face recognition. HighlightsThis paper proposes data-driven techniques for smoothing LBP histograms.The proposed smoothing techniques cover unsupervised and supervised variants.The techniques are evaluated on material categorization and face recognition.Histogram smoothing is beneficial especially in small-sample-size scenarios.

Martians from outer space: experimenting with location-aware cooperative multiplayer gaming on public displays

In this paper, we describe our experiences with location-aware cooperative multiplayer game on public displays. The game world is modelled after the city of Oulu, Finland where players protect the city from a Martian invasion. We investigate the potential of the used platform, the effects of locality and how a more complex and challenging gaming experience on public displays is received. We demonstrate that locality does have a significant effect on the game-play especially when the player can actually see the familiar surroundings in the game world. We also show that while the use of the different services vary a lot from place to place, our game can maintain a very good ranking when compared to other, more casual games.