Hamed Kiani Galoogahi

Multi-channel Correlation Filters

Modern descriptors like HOG and SIFT are now commonly used in vision for pattern detection within image and video. From a signal processing perspective, this detection process can be efficiently posed as a correlation/ convolution between a multi-channel image and a multi-channel detector/filter which results in a single channel response map indicating where the pattern (e.g. object) has occurred. In this paper, we propose a novel framework for learning a multi-channel detector/filter efficiently in the frequency domain, both in terms of training time and memory footprint, which we refer to as a multichannel correlation filter. To demonstrate the effectiveness of our strategy, we evaluate it across a number of visual detection/ localization tasks where we: (i) exhibit superior performance to current state of the art correlation filters, and (ii) superior computational and memory efficiencies compared to state of the art spatial detectors.

Correlation filters with limited boundaries

Correlation filters take advantage of specific properties in the Fourier domain allowing them to be estimated efficiently: O(N D log D) in the frequency domain, versus O(D3 + N D2) spatially where D is signal length, and N is the number of signals. Recent extensions to correlation filters, such as MOSSE, have reignited interest of their use in the vision community due to their robustness and attractive computational properties. In this paper we demonstrate, however, that this computational efficiency comes at a cost. Specifically, we demonstrate that only 1/D proportion of shifted examples are unaffected by boundary effects which has a dramatic effect on detection/tracking performance. In this paper, we propose a novel approach to correlation filter estimation that: (i) takes advantage of inherent computational redundancies in the frequency domain, (ii) dramatically reduces boundary effects, and (iii) is able to implicitly exploit all possible patches densely extracted from training examples during learning process. Impressive object tracking and detection results are presented in terms of both accuracy and computational efficiency.

Inter-modality Face Sketch Recognition

Automatic face sketch recognition plays an important role in law enforcement. Recently, various methods have been proposed to address the problem of face sketch recognition by matching face photos and sketches, which are of different modalities. However, their performance is strongly affected by the modality difference between sketches and photos. In this paper, we propose a new face descriptor based on gradient orientations to reduce the modality difference in feature extraction stage, called Histogram of Averaged Oriented Gradients (HAOG). Experiments on CUFS database show that the new descriptor outperforms the state-of-the-art approaches.

Face sketch recognition by Local Radon Binary Pattern: LRBP

In this paper, we propose a new face descriptor to directly match face photos and sketches of different modalities, called Local Radon Binary Pattern (LRBP). LRBP is inspired by the fact that the shape of a face photo and its corresponding sketch is similar, even when the sketch is exaggerated by an artist. Therefore, the shape of face can be exploited to compute features which are robust against modality differences between face photo and sketch. In LRBP framework, the characteristics of face shape are captured by transforming face image into Radon space. Then, micro-information of face shape in new space is encoded by Local Binary Pattern (LBP). Finally, LRBP is computed by concatenating histograms of local LBPs. In order to capture both local and global characteristics of face shape, LRBP is extracted in a spatial pyramid fashion. Experiments on CUFS and CUFSF datasets indicate the efficiency of LRBP for face sketch recognition.

Shape Classification Using Local and Global Features

In this paper, we address the shape classification problem by proposing a new integrating approach for shape classification that gains both local and global image representation using Histogram of Oriented Gradient (HOG). In both local and global feature extraction steps, we use PCA to make this method invariant to shapes rotation. Moreover, by using a learning algorithm based on Adaboost we improve the global feature extraction by selecting a small number of more discriminative visual features through a large raw visual features set to increase the classification accuracy. Our local method is adopted from the popular bag of key points approach for shape classification. To integrate the classification results generated based on both local and global features, we use a combining classifier to perform the final classification for a new unknown image query. The experiment results show that this new method achieves the state-of-art accuracy for shape classification on the animal dataset in [8].

Abnormality Detection with Improved Histogram of Oriented Tracklets

Recently the histogram of oriented tracklets (HOT) was shown to be an efficient video representation for abnormality detection and achieved state-of-the-arts on the available datasets. Unlike standard video descriptors that mainly employ low level motion features, e.g. optical flow, the HOT descriptor simultaneously encodes magnitude and orientation of tracklets as a mid-level representation over crowd motions. However, extracting tracklets in HOT suffers from poor salient point initialization and tracking drift in the presence of occlusion. Moreover, count-based HOT histogramming does not properly take into account the motion characteristics of abnormal motions. This paper extends the HOT by addressing these drawbacks introducing an enhanced version of HOT, named Improved HOT. First, we propose to initialize salient points in each frame instead of the first frame, as the HOT does. Second, we replace the naive count-based histogramming by the richer statistics of crowd movement (i.e., motion distribution). The evaluation of the Improved HOT on different datasets, namely UCSD, BEHAVE and UMN, yields compelling results in abnormality detection, by outperforming the original HOT and the state-of-the-art descriptors based on optical flow, dense trajectories and the social force models.

Face photo retrieval by sketch example

Face photo-sketch matching has received great attention in recent years due to its vital role in law enforcement. The major challenge of matching face photo and sketch is difference of visual characteristics between face photo and sketch which is referred as modality gap. Earlier approaches have reduced the modality gap by synthesizing face photos and sketches in a same modality (photo or sketch). However, the effectiveness of these approaches is highly affected by synthesis results. That means a poor synthesis might degrade the performance of matching. Therefore, recent works have focused to directly match face photo and sketch of different modalities. However, the features used by these approaches are not robust against modality gap. In this paper, a modality-invariant face descriptor called Gabor Shape is proposed to retrieve face photos based on a probe sketch. Experiments on CUFS and CUFSF datasets show that the new descriptor outperforms the state-of-the-art approaches.

Detecting Abnormal Behavioral Patterns in Crowd Scenarios

This Chapter presents a framework for the the task of abnormality detection in crowded scenes based on the analysis of trajectories, build up upon a novel video descriptor, called Histogram of Oriented Tracklets. Unlike standard approaches that employ low level motion features, e.g. optical flow, to form video descriptors, we propose to exploit mid-level features extracted from long-range motion trajectories called tracklets, which have been successfully applied for action modeling and video analysis. Following standard procedure, a video sequence is divided into spatio-temporal cuboids within which we collect statistics of the tracklets passing through them. Specifically, tracklets orientation and magnitude are quantized in a two-dimensional histogram which encodes the actual motion patterns in each cuboid. These histograms are then fed into machine learning models (e.g., Latent Dirichlet allocation and Support Vector Machines) to detect abnormal behaviors in video sequences. The evaluation of the proposed descriptor on different datasets, namely UCSD, BEHAVE, UMN and Violence in Crowds, yields compelling results in abnormality detection, by setting new state-of-the-art and outperforming former descriptors based on the optical flow, dense trajectories and social force models.

Tracking Groups of People in Presence of Occlusion

This paper addresses the problem of people group tracking in presence of occlusion as people form groups, interact within groups or leave groups. Foreground objects (a person or a group of people) from two consecutive frames are matched based on appearance (RGB histogram) and object location (2D region) similarity. While tracking, this method determines and handles some events such as objects merging and splitting using forward and backward matching matrices. The experimental results show that the proposed algorithm is efficient to track group of people in cluttered and complex environments even when total or partial occlusion occurs.

Person re-identification using multiple first-person-views on wearable devices

The rise of wearable devices has led to many new ways of re-identifying an individual. Unlike static cameras, where the views are often restricted or zoomed out and occlusions are common scenarios, first-person-views (FPVs) or ego-centric views see people closely and mostly get un-occluded face images. In this paper, we propose a face re-identification framework designed for a network of multiple wearable devices. This framework utilizes a global data association method termed as Network Consistent Reidentification (NCR) that not only helps in maintaining consistency in association results across the network, but also improves the pair-wise face re-identification accuracy. To test the proposed pipeline, we collected a database of FPV videos of 72 persons using multiple wearable devices (such as Google Glasses) in a multi-storied office environment. Experimental results indicate that NCR is able to consistently achieve large performance gains when compared to the state-of-the-art methodologies.