João C. Neves

Biometric recognition in surveillance scenarios: a survey

Interest in the security of individuals has increased in recent years. This increase has in turn led to much wider deployment of surveillance cameras worldwide, and consequently, automated surveillance systems research has received more attention from the scientific community than before. Concurrently, biometrics research has become more popular as well, and it is supported by the increasing number of approaches devised to address specific degradation factors of unconstrained environments. Despite these recent efforts, no automated surveillance system that performs reliable biometric recognition in such an environment has become available. Nevertheless, recent developments in human motion analysis and biometric recognition suggest that both can be combined to develop a fully automated system. As such, this paper reviews recent advances in both areas, with a special focus on surveillance scenarios. When compared to previous studies, we highlight two distinct features, i.e., (1) our emphasis is on approaches that are devised to work in unconstrained environments and surveillance scenarios; and (2) biometric recognition is the final goal of the surveillance system, as opposed to behavior analysis, anomaly detection or action recognition.

Acquiring high-resolution face images in outdoor environments: A master-slave calibration algorithm

Facial recognition at-a-distance in surveillance scenarios remains an open problem, particularly due to the small number of pixels representing the facial region. The use of pan-tilt-zoom (PTZ) cameras has been advocated to solve this problem, however, the existing approaches either rely on rough approximations or additional constraints to estimate the mapping between image coordinates and pan-tilt parameters. In this paper, we aim at extending PTZ-assisted facial recognition to surveillance scenarios by proposing a master-slave calibration algorithm capable of accurately estimating pan-tilt parameters without depending on additional constraints. Our approach exploits geometric cues to automatically estimate subjects height and thus determine their 3D position. Experimental results show that the presented algorithm is able to acquire high-resolution face images at a distance ranging from 5 to 40 meters with high success rate. Additionally, we certify the applicability of the aforementioned algorithm to biometric recognition through a face recognition test, comprising 20 probe subjects and 13,020 gallery subjects.

Dynamic camera scheduling for visual surveillance in crowded scenes using Markov random fields

The use of pan-tilt-zoom (PTZ) cameras for capturing high-resolution data of human-beings is an emerging trend in surveillance systems. However, this new paradigm entails additional challenges, such as camera scheduling, that can dramatically affect the performance of the system. In this paper, we present a camera scheduling approach capable of determining - in real-time - the sequence of acquisitions that maximizes the number of different targets obtained, while minimizing the cumulative transition time. Our approach models the problem as an undirected graphical model (Markov random field, MRF), which energy minimization can approximate the shortest tour to visit the maximum number of targets. A comparative analysis with the state-of-the-art camera scheduling methods evidences that our approach is able to improve the observation rate while maintaining a competitive tour time.

Segmenting the periocular region using a hierarchical graphical model fed by texture / shape information and geometrical constraints

Using the periocular region for biometric recognition is an interesting possibility: this area of the human body is highly discriminative among subjects and relatively stable in appearance. In this paper, the main idea is that improved solutions for defining the periocular region-of-interest and better pose / gaze estimates can be obtained by segmenting (labelling) all the components in the periocular vicinity. Accordingly, we describe an integrated algorithm for labelling the periocular region, that uses a unique model to discriminate between seven components in a single-shot: iris, sclera, eyelashes, eyebrows, hair, skin and glasses. Our solution fuses texture / shape descriptors and geometrical constraints to feed a two-layered graphical model (Markov Random Field), which energy minimization provides a robust solution against uncontrolled lighting conditions and variations in subjects pose and gaze.

Quis-Campi: Extending in the Wild Biometric Recognition to Surveillance Environments

Efforts in biometrics are being held into extending robust recognition techniques to in the wild scenarios. Nonetheless, and despite being a very attractive goal, human identification in the surveillance context remains an open problem. In this paper, we introduce a novel biometric system – Quis-Campi – that effectively bridges the gap between surveillance and biometric recognition while having a minimum amount of operational restrictions. We propose a fully automated surveillance system for human recognition purposes, attained by combining human detection and tracking, further enhanced by a PTZ camera that delivers data with enough quality to perform biometric recognition. Along with the system concept, implementation details for both hardware and software modules are provided, as well as preliminary results over a real scenario.

FTP@VDTN — A file transfer application for Vehicular Delay-Tolerant Networks

Vehicular Delay-Tolerant Networks (VDTNs) aim to provide non-real time services and applications, such as electronic mail or file transfer, in environments with sparse and intermittent connectivity, variable delays, or where an end-to-end connection may not exist. Gathering contributions from opportunistic and cooperative networks, VDTN tries to generalize the delay-tolerant networks concept and apply it to vehicular networks. In VDTNs, data is carried between network nodes using vehicles that allow network connectivity. This paper presents a file transfer application for VDTNs, called FTP@VDTN, and studies its performance through a laboratory VDTN testbed using two routing mechanisms (Epidemic, and Spray and Wait) combined with different scheduling and dropping policies. It was demonstrated that FTP@VDTN works properly in a VDTN testbed. In terms of performance evaluation of VDTNs using FTP@VDTN, it was shown that remaining lifetime combination of scheduling and dropping policies perform better for both routing schemes.

Joint Head Pose/Soft Label Estimation for Human Recognition In-The-Wild

Soft biometrics have been emerging to complement other traits and are particularly useful for poor quality data. In this paper, we propose an efficient algorithm to estimate human head poses and to infer soft biometric labels based on the 3D morphology of the human head. Starting by considering a set of pose hypotheses, we use a learning set of head shapes synthesized from anthropometric surveys to derive a set of 3D head centroids that constitutes a metric space. Next, representing queries by sets of 2D head landmarks, we use projective geometry techniques to rank efficiently the joint 3D head centroids/pose hypotheses according to their likelihood of matching each query. The rationale is that the most likely hypotheses are sufficiently close to the query, so a good solution can be found by convex energy minimization techniques. Once a solution has been found, the 3D head centroid and the query are assumed to have similar morphology, yielding the soft label. Our experiments point toward the usefulness of the proposed solution, which can improve the effectiveness of face recognizers and can also be used as a privacy-preserving solution for biometric recognition in public environments.

A Calibration Algorithm for Multi-camera Visual Surveillance Systems Based on Single-View Metrology

The growing concerns about persons security and the increasing popularity of pan-tilt-zoom (PTZ) cameras, have been raising the interest on automated master-slave surveillance systems. Such systems are typically composed by (1) a fixed wide-angle camera that covers a large area, detects and tracks moving objects in the scene; and (2) a PTZ camera, that provides a close-up view of an object of interest. Previously published approaches attempted to establish 2D correspondences between the video streams of both cameras, which is a ill-posed formulation due to the absence of depth information. On the other side, 3D-based approaches are more accurate but require more than one fixed camera to estimate depth information. In this paper, we describe a novel method for easy and precise calibration of a master-slave surveillance system, composed by a single fixed wide-angle camera. Our method exploits single view metrology to infer 3D data of the tracked humans and to self-perform the transformation between camera views. Experimental results in both simulated and realistic scenes point for the effectiveness of the proposed model in comparison with the state-of-the-art.

A Master-Slave Calibration Algorithm with Fish-Eye Correction

Surveillance systems capable of autonomously monitoring vast areas are an emerging trend, particularly when wide-angle cameras are combined with pan-tilt-zoom (PTZ) cameras in a master-slave configuration. The use of fish-eye lenses allows the master camera to maximize the coverage area while the PTZ acts as a foveal sensor, providing high-resolution images of regions of interest. Despite the advantages of this architecture, the mapping between image coordinates and pan-tilt values is the major bottleneck in such systems, since it depends on depth information and fish-eye effect correction. In this paper, we address these problems by exploiting geometric cues to perform height estimation. This information is used both for inferring 3D information from a single static camera deployed on an arbitrary position and for determining lens parameters to remove fish-eye distortion. When compared with the previous approaches, our method has the following advantages: (1) fish-eye distortion is corrected without relying on calibration patterns; (2) 3D information is inferred from a single static camera disposed on an arbitrary location of the scene.

Creating synthetic IrisCodes to feed biometrics experiments

The collection of iris data suitable to be used in experiments is difficult, mainly due to two factors: 1) the time spent by volunteers in the acquisition process; and 2) security / privacy concerns of volunteers. Even though there are methods to create images of artificial irises, there is no method exclusively focused in the synthesis of the iris biometric signatures (IrisCodes). In experiments related with some phases of the biometric recognition process (e.g., indexing / retrieval), a large number of signatures is required for proper evaluation, which, in case of real data, is extremely hard to obtain. Hence, this paper describes a stochastic method to synthesize IrisCodes, based on the notion of data correlation. These artificial signatures can be used to feed experiments on iris recognition, namely on the iris matching, indexing and retrieval phases. We experimentally confirmed that both the genuine and impostor distributions obtained on the artificial data closely resemble the values obtained in data sets of real irises. Finally, another interesting feature is that the method is easily parametrized to mimic IrisCodes extracted from data of varying levels of quality, i.e., ranging from data acquired in high controlled to unconstrained environments.