Iffat Zafar

Real-time automatic license plate recognition for CCTV forensic applications

We propose an efficient real-time automatic license plate recognition (ALPR) framework, particularly designed to work on CCTV video footage obtained from cameras that are not dedicated to the use in ALPR. At present, in license plate detection, tracking and recognition are reasonably well-tackled problems with many successful commercial solutions being available. However, the existing ALPR algorithms are based on the assumption that the input video will be obtained via a dedicated, high-resolution, high-speed camera and is/or supported by a controlled capture environment, with appropriate camera height, focus, exposure/shutter speed and lighting settings. However, typical video forensic applications may require searching for a vehicle having a particular number plate on noisy CCTV video footage obtained via non-dedicated, medium-to-low resolution cameras, working under poor illumination conditions. ALPR in such video content faces severe challenges in license plate localization, tracking and recognition stages. This paper proposes a novel approach for efficient localization of license plates in video sequence and the use of a revised version of an existing technique for tracking and recognition. A special feature of the proposed approach is that it is intelligent enough to automatically adjust for varying camera distances and diverse lighting conditions, a requirement for a video forensic tool that may operate on videos obtained by a diverse set of unspecified, distributed CCTV cameras.

Two-dimensional statistical linear discriminant analysis for real-time robust vehicle-type recognition

Automatic vehicle Make and Model Recognition (MMR) systems provide useful performance enhancements to vehicle recognitions systems that are solely based on Automatic License Plate Recognition (ALPR) systems. Several car MMR systems have been proposed in literature. However these approaches are based on feature detection algorithms that can perform sub-optimally under adverse lighting and/or occlusion conditions. In this paper we propose a real time, appearance based, car MMR approach using Two Dimensional Linear Discriminant Analysis that is capable of addressing this limitation. We provide experimental results to analyse the proposed algorithms robustness under varying illumination and occlusions conditions. We have shown that the best performance with the proposed 2D-LDA based car MMR approach is obtained when the eigenvectors of lower significance are ignored. For the given database of 200 car images of 25 different make-model classifications, a best accuracy of 91% was obtained with the 2D-LDA approach. We use a direct Principle Component Analysis (PCA) based approach as a benchmark to compare and contrast the performance of the proposed 2D-LDA approach to car MMR. We conclude that in general the 2D-LDA based algorithm supersedes the performance of the PCA based approach.

Localized contourlet features in vehicle make and model recognition

Automatic vehicle Make and Model Recognition (MMR) systems provide useful performance enhancements to vehicle recognitions systems that are solely based on Automatic Number Plate Recognition (ANPR) systems. Several vehicle MMR systems have been proposed in literature. In parallel to this, the usefulness of multi-resolution based feature analysis techniques leading to efficient object classification algorithms have received close attention from the research community. To this effect, Contourlet transforms that can provide an efficient directional multi-resolution image representation has recently been introduced. Already an attempt has been made in literature to use Curvelet/Contourlet transforms in vehicle MMR. In this paper we propose a novel localized feature detection method in Contourlet transform domain that is capable of increasing the classification rates up to 4%, as compared to the previously proposed Contourlet based vehicle MMR approach in which the features are non-localized and thus results in sub-optimal classification. Further we show that the proposed algorithm can achieve the increased classification accuracy of 96% at significantly lower computational complexity due to the use of Two Dimensional Linear Discriminant Analysis (2DLDA) for dimensionality reduction by preserving the features with high between-class variance and low inter-class variance.

Carried Object Detection in Videos Using Color Information

Automatic baggage detection has become a subject of significant practical interest in recent years. In this paper, we propose an approach to baggage detection in CCTV video footage that uses color information to address some of the vital shortcomings of state-of-the-art algorithms. The proposed approach consists of typical steps used in baggage detection, namely, the estimation of moving direction of humans carrying baggage, construction of human-like temporal templates, and their alignment with the best matched view-specific exemplars. In addition, we utilize the color information to define the region that most likely belongs to a human torso in order to reduce the false positive detections. A key novel contribution is the persons viewing direction estimation using machine learning and shoulder shape related features. Further enhancement of baggage detection and segmentation is achieved by exploiting the CIELAB color space properties. The proposed system has been extensively tested for its effectiveness, at each stage of improvement, on PETS 2006 dataset and additional CCTV video footage captured to cover specific test scenarios. The experimental results suggest that the proposed algorithm is capable of superseding the functional performance of state-of-the-art baggage detection algorithms.

Human silhouette extraction on FPGAs for infrared night vision military surveillance

Infrared visual surveillance has become an important mean to secure military camps, reassure soldier security, and detect suspected terror activities in the battle fields. An intelligent infrared surveillance system is aimed to provide real-time intelligent analysis of the perceived scene and find out human targets instantly to assist the soldier/commanders to make the right decision in a just-in-time mode to save our soldiers from life risks. To attain this, automatic detection of moving human objects from the scene is an essential step. In this paper, we present an FPGAs-based architecture to perform on-chip human silhouette extraction using a parallel architecture with systolic arrays. The architecture is designed in VHDL and simulated with real FLIR videos. The experiment shows that the designed processor on FPGAs can efficiently extract the human contour instantly from infrared videos, which exhibits great potential to facilitate the further analysis of the battlefield scenes for military-purpose surveillance systems.

Real time multiple two dimensional barcode reader

The proposed work describes a system that can be used to detect and decode multiple Two-Dimensional (2D) barcodes (i.e. Datamatrix). The proposed system is capable of detecting and decoding of barcodes covered by polyethylene foiling in moving images. The foiling causes reflection on the captured images and hence provides challenges to detection/decoding. Similarly the proposed system addresses the problem of poor lighting conditions. The main contribution is the design of an optical hardware setup that is integrated with machine vision algorithms to successfully address the reflection and illumination challenges. A recognition accuracy of approximately 98% is achieved.

Human object annotation for surveillance video forensics

Abstract. A system that can automatically annotate surveillance video in a manner useful for locating a person with a given description of clothing is presented. Each human is annotated based on two appearance features: primary colors of clothes and the presence of text/logos on clothes. The annotation occurs after a robust foreground extraction stage employing a modified Gaussian mixture model-based approach. The proposed pipeline consists of a preprocessing stage where color appearance of an image is improved using a color constancy algorithm. In order to annotate color information for human clothes, we use the color histogram feature in HSV space and find local maxima to extract dominant colors for different parts of a segmented human object. To detect text/logos on clothes, we begin with the extraction of connected components of enhanced horizontal, vertical, and diagonal edges in the frames. These candidate regions are classified as text or nontext on the basis of their local energy-based shape histogram features. Further, to detect humans, a novel technique has been proposed that uses contourlet transform-based local binary pattern (CLBP) features. In the proposed method, we extract the uniform direction invariant LBP feature descriptor for contourlet transformed high-pass subimages from vertical and diagonal directional bands. In the final stage, extracted CLBP descriptors are classified by a trained support vector machine. Experimental results illustrate the superiority of our method on large-scale surveillance video data.

Human object articulation for CCTV video forensics

In this paper we present a system which is focused on developing algorithms for automatic annotation/articulation of humans passing through a surveillance camera in a way useful for describing a person/criminal by a crime scene witness. Each human is articulated/annotated based on two appearance features: 1. primary colors of clothes in the head, body and legs region. 2. presence of text/logo on the clothes. The annotation occurs after a robust foreground extraction based on a modified approach to Gaussian Mixture model and detection of human from segmented foreground images. The proposed pipeline consists of a preprocessing stage where we improve color quality of images using a basic color constancy algorithm and further improve the results using a proposed post-processing method. The results show a significant improvement to the illumination of the video frames. In order to annotate color information for human clothes, we apply 3D Histogram analysis (with respect to Hue, Saturation and Value) on HSV converted image regions of human body parts along with extrema detection and thresholding to decide the dominant color of the region. In order to detect text/logo on the clothes as another feature to articulate humans, we begin with the extraction of connected components of enhanced horizontal, vertical and diagonal edges in the frames. These candidate regions are classified as text or non-text on the bases of their Local Energy based Shape Histogram (LESH) features combined with KL divergence as classification criteria. To detect humans, a novel technique has been proposed that uses a combination of Histogram of Oriented Gradients (HOG) and Contourlet transform based Local Binary Patterns (LBP) with Adaboost as classifier. Initial screening of foreground objects is performed by using HOG features. To further eliminate the false positives due to noise form background and improve results, we apply Contourlet-LBP feature extraction on the images. In the proposed method, we extract the LBP feature descriptor for Contourlet transformed high pass sub-images from vertical and diagonal directional bands. In the final stage, extracted Contourlet-LBP descriptors are applied to Adaboost for classification. The proposed frame work showed fairly fine performance when tested on a CCTV test dataset.

Real-time multi-barcode reader for industrial applications

The advances in automated production processes have resulted in the need for detecting, reading and decoding 2D datamatrix barcodes at very high speeds. This requires the correct combination of high speed optical devices that are capable of capturing high quality images and computer vision algorithms that can read and decode the barcodes accurately. Such barcode readers should also be capable of resolving fundamental imaging challenges arising from blurred barcode edges, reflections from possible polyethylene wrapping, poor and/or non-uniform illumination, fluctuations of focus, rotation and scale changes. Addressing the above challenges in this paper we propose the design and implementation of a high speed multi-barcode reader and provide test results from an industrial trial. To authors knowledge such a comprehensive system has not been proposed and fully investigated in existing literature. To reduce the reflections on the images caused due to polyethylene wrapping used in typical packaging, polarising filters have been used. The images captured using the optical system above will still include imperfections and variations due to scale, rotation, illumination etc. We use a number of novel image enhancement algorithms optimised for use with 2D datamatrix barcodes for image de-blurring, contrast point and self-shadow removal using an affine transform based approach and non-uniform illumination correction. The enhanced images are subsequently used for barcode detection and recognition. We provide experimental results from a factory trial of using the multi-barcode reader and evaluate the performance of each optical unit and computer vision algorithm used. The results indicate an overall accuracy of 99.6 % in barcode recognition at typical speeds of industrial conveyor systems.

REAL-TIME SPEAKER IDENTIFICATION FOR VIDEO CONFERENCING

Automatic speaker identification in a videoconferencing environment will allow conference attendees to focus their attention on the conference rather than having to be engaged manually in identifying which channel is active and who may be the speaker within that channel. In this work we present a real-time, audio-coupled video based approach to address this problem, but focus more on the video analysis side. The system is driven by the need for detecting a talking human via the use of computer vision algorithms. The initial stage consists of a face detector which is subsequently followed by a lip-localization algorithm that segments the lip region. A novel approach for lip movement detection based on image registration and using the Coherent Point Drift (CPD) algorithm is proposed. Coherent Point Drift (CPD) is a technique for rigid and non-rigid registration of point sets. We provide experimental results to analyse the performance of the algorithm when used in monitoring real life videoconferencing data.