Banafshe Arbab-Zavar

The ear as a biometric

It is more than 10 years since the first tentative experiments in ear biometrics were conducted and it has now reached the “adolescence” of its development towards a mature biometric. Here we present a timely retrospective of the ensuing research since those early days. Whilst its detailed structure may not be as complex as the iris we show that the ear has unique security advantages over other biometrics. It is most unusual even unique in that it supports not only visual and forensic recognition but also acoustic recognition at the same time. This together with its deep three-dimensional structure and its robust resistance to change with age will make it very difficult to counterfeit thus ensuring that the ear will occupy a special place in situations requiring a high degree of protection.

On shape-mediated enrolment in ear biometrics

Ears are a new biometric with major advantage in that they appear to maintain their shape with increased age. Any automatic biometric system needs enrolment to extract the target area from the background. In ear biometrics the inputs are often human head profile images. Furthermore ear biometrics is concerned with the effects of partial occlusion mostly caused by hair and earrings. We propose an ear enrolment algorithm based on finding the elliptical shape of the ear using a Hough Transform (HT) accruing tolerance to noise and occlusion. Robustness is improved further by enforcing some prior knowledge. We assess our enrolment on two face profile datasets; as well as synthetic occlusion.

On Model-Based Analysis of Ear Biometrics

Ears are a new biometric with major advantage in that they appear to maintain their structure with increasing age. Most current approaches are holistic and describe the ear by its general properties. We propose a new model-based approach, capitalizing on explicit structure and with the advantages of being robust in noise and occlusion. Our model is a constellation of generalized ear parts, which is learned off-line using an unsupervised learning algorithm over an enrolled training set of 63 ear images. The Scale Invariant Feature Transform (SIFT), is used to detect the features within the ear images. In recognition, given a profile image of the human head, the ear is enrolled and recognised from the parts selected via the model. We achieve an encouraging recognition rate, on an image database selected from the XM2VTS database. A head-to-head comparison with PCA is also presented to show the advantage derived by the use of the model in successful occlusion handling.

On guided model-based analysis for ear biometrics

Ears are a new biometric with a major advantage in that they appear to maintain their structure with increasing age. Current approaches have exploited 2D and 3D images of the ear in human identification. Contending that the ear is mainly a planar shape we use 2D images, which are consistent with deployment in surveillance and other planar-image scenarios. So far ear biometric approaches have mostly capitalized on general properties and overall appearance of ear images, and the details of the ear structure have been little discussed. Using the embryological studies of the ear development, which reveal a component-wise structure for the ear, we propose a new model-based approach. Our model is a part-wise description of the ear derived by a stochastic clustering on a set of scale invariant features of a training set. We further extend our model description, by a wavelet-based analysis with a specific aim of capturing information in the ears boundary structures, which can augment discriminant variability. In recognition, ears are automatically enroled and then recognized via the parts selected by the model. The incorporation of the wavelet-based analysis of the outer ear structures forms an extended or hybrid method. By results, both in modelling and recognition, our new model-based approach does indeed appear to be a promising new approach to ear biometrics. Recognizing the occlusion by hair as one of the main obstacles hindering the deployment of ear biometrics, we have specifically chosen our techniques to provide performance advantages in occlusion. We shall present a thorough evaluation of performance in occlusion, using a robust PCA for comparison purposes. Our new hybrid method does indeed appear to be a promising new approach to ear biometrics, by guiding a model-based analysis via anatomical knowledge.

Robust log-Gabor filter for ear biometrics

Ears are a new biometric with major advantage in that they appear to maintain their structure with increasing age. Expanding on our previous parts-based model, we propose a new wavelet approach. In this, the log-Gabor filter exploits the frequency content of the ear boundary curves. Extending our model description, a specific aim of the new approach is to capture information in the ear¿s outer structures. Ear biometrics is also concerned with the effects of partial occlusion, mostly by hair and earrings. By localization, intuitively a wavelet can offer performance advantage when handling occluded data. We also add a more robust matching strategy to restrict the influence of erroneous wavelet coefficients. Significant improvement is observed when we combine the model and the log-Gabor filter, and we will show that this improvement is maintained as the ears get occluded.

Multi-modal computer vision for the detection of multi-scale crowd physical motions and behavior in confined spaces

Crowd physical motion and behaviour detection during evacuation from confined spaces using computer vision is the main focus of research in the eVACUATE project. Its early foundations and development perspectives are discussed in this paper. Specifically, the main target in our development is to achieve good rates of correct detection and classification of crowd motion and behaviour in confined spaces respectively. However, the performance of the computer vision algorithms, which are put in place for the detection of crowd motion and behaviour, greatly depends on the quality, including causality, of the multi-modal observation data with ground truth. Furthermore, it is of paramount importance to take into account contextual information about the confined spaces concerned in order to confirm the type of detected behaviours. The pilot venues for crowd evacuation experimentations include: (1) Athens International Airport, Greece; (2) An underground train station in Bilbao, Spain; (3) A stadium in San Sebastian, Spain; and (4) A large cruise ship in St. Nazaire, France.

On supervised human activity analysis for structured environments

We consider the problem of developing an automated visual solution for detecting human activities within industrial environments. This has been performed using an overhead view. This view was chosen over more conventional oblique views as it does not suffer from occlusion, but still retains powerful cues about the activity of individuals. A simple blob tracker has been used to track the most significant moving parts i.e. human beings. The output of the tracking stage was manually labelled into 4 distinct categories: walking; carrying; handling and standing still which are taken together from the basic building blocks of a higher work flow description. These were used to train a decision tree using one subset of the data. A separate training set is used to learn the patterns in the activity sequences by Hidden Markov Models (HMM). On independent testing, the HMM models are applied to analyse and modify the sequence of activities predicted by the decision tree.

Context Ontology Modelling for Improving Situation Awareness and Crowd Evacuation from Confined Spaces

Crowd evacuation management at large venues such as airports, stadiums, cruise ships or metro stations requires the deployment and access to a Common Operational Picture (COP) of the venue, with real-time intelligent contextual interpretation of crowd behaviour. Large CCTV and sensor network feeds all provide important but heterogeneous observations about crowd safety at the venue of interest. Hence, these observations must be critically analyzed and interpreted for supporting security managers of crowd safety at venues. Specifically, the large volume of the generated observations needs to be interpreted in context of the venue operational grounds, crowd-gathering event times and the knowledge on crowd expected behaviour. In this paper, a new context ontology modelling approach is introduced. It is based on knowledge about venue background information, expected crowd behaviours and their manifested features of observations. The aim is to improve situation awareness about crowd safety in crisis management and decision-support.

On hierarchical modelling of motion for workflow analysis from overhead view

Understanding human behaviour is a high level perceptual problem, one which is often dominated by the contextual knowledge of the environment, and where concerns such as occlusion, scene clutter and high within-class variations are commonplace. Nonetheless, such understanding is highly desirable for automated visual surveillance. We consider this problem in a context of a workflow analysis within an industrial environment. The hierarchical nature of the workflow is exploited to split the problem into ‘activity’ and ‘task’ recognition. In this, sequences of low level activities are examined for instances of a task while the remainder are labelled as background. An initial prediction of activity is obtained using shape and motion based features of the moving blob of interest. A sequence of these activities is further adjusted by a probabilistic analysis of transitions between activities using hidden Markov models (HMMs). In task detection, HMMs are arranged to handle the activities within each task. Two separate HMMs for task and background compete for an incoming sequence of activities. Imagery derived from a camera mounted overhead the target scene has been chosen over the more conventional oblique views (from the side) as this view does not suffer from as much occlusion, and it poses a manageable detection and tracking problem while still retaining powerful cues as to the workflow patterns. We evaluate our approach both in activity and task detection on a challenging dataset of surveillance of human operators in a car manufacturing plant. The experimental results show that our hierarchical approach can automatically segment the timeline and spatially localize a series of predefined tasks that are performed to complete a workflow.

Tools for semi-automatic monitoring of industrial workflows

This paper describes a tool chain for monitoring complex workflows. Statistics obtained from automatic workflow monitoring in a car assembly environment assist in improving industrial safety and process quality. To this end, we propose automatic detection and tracking of humans and their activity in multiple networked cameras. The described tools offer human operators retrospective analysis of a huge amount of pre-recorded and analyzed footage from multiple cameras in order to get a comprehensive overview of the workflows. Furthermore, the tools help technical administrators in adjusting algorithms by letting the user correct detections (for relevance feedback) and ground truth for evaluation. Another important feature of the tool chain is the capability to inform the employees about potentially risky conditions using the tool for automatic detection of unusual scenes.