Navneet Dalal

Histograms of oriented gradients for human detection

We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.

Human detection using oriented histograms of flow and appearance

Detecting humans in films and videos is a challenging problem owing to the motion of the subjects, the camera and the background and to variations in pose, appearance, clothing, illumination and background clutter. We develop a detector for standing and moving people in videos with possibly moving cameras and backgrounds, testing several different motion coding schemes and showing empirically that orientated histograms of differential optical flow give the best overall performance. These motion-based descriptors are combined with our Histogram of Oriented Gradient appearance descriptors. The resulting detector is tested on several databases including a challenging test set taken from feature films and containing wide ranges of pose, motion and background variations, including moving cameras and backgrounds. We validate our results on two challenging test sets containing more than 4400 human examples. The combined detector reduces the false alarm rate by a factor of 10 relative to the best appearance-based detector, for example giving false alarm rates of 1 per 20,000 windows tested at 8% miss rate on our Test Set 1.

The 2005 PASCAL visual object classes challenge

The PASCAL Visual Object Classes Challenge ran from February to March 2005. The goal of the challenge was to recognize objects from a number of visual object classes in realistic scenes (i.e. not pre-segmented objects). Four object classes were selected: motorbikes, bicycles, cars and people. Twelve teams entered the challenge. In this chapter we provide details of the datasets, algorithms used by the teams, evaluation criteria, and results achieved.

A Robust Algorithm for Characterizing Anisotropic Local Structures

This paper proposes a robust estimation and validation framework for characterizing local structures in a positive multi-variate continuous function approximated by a Gaussian-based model. The new solution is robust against data with large deviations from the model and margin-truncations induced by neighboring structures. To this goal, it unifies robust statistical estimation for parametric model fitting and multi-scale analysis based on continuous scale-space theory. The unification is realized by formally extending the mean shift-based density analysis towards continuous signals whose local structure is characterized by an anisotropic fully-parameterized covariance matrix. A statistical validation method based on analyzing residual error of the chi-square fitting is also proposed to complement this estimation framework. The strength of our solution is the aforementioned robustness. Experiments with synthetic 1D and 2D data clearly demonstrate this advantage in comparison with the γ-normalized Laplacian approach [12] and the standard sample estimation approach [13, p.179]. The new framework is applied to 3D volumetric analysis of lung tumors. A 3D implementation is evaluated with high-resolution CT images of 14 patients with 77 tumors, including 6 part-solid or ground-glass opacity nodules that are highly non-Gaussian and clinically significant. Our system accurately estimated 3D anisotropic spread and orientation for 82% of the total tumors and also correctly rejected all the failures without any false rejection and false acceptance. This system processes each 32-voxel volume-of-interest by an average of two seconds with a 2.4GHz Intel CPU. Our framework is generic and can be applied for the analysis of blob-like structures in various other applications.

Indexing key positions between multiple videos

Given two or more video sequences containing similar human activities (running, jumping, etc.), we want to devise a method which extracts spatio-temporal signatures associated with these activities, compares these signatures, and aligns key positions from different videos. We introduce a method which, in conjunction with a number of hypotheses, allows the analysis of the motion of specific body parts and extracts their 2D (image plane) time-varying trajectories as well as their 3D trajectories. Two such trajectories recovered from two different videos have different characteristics. We develop a curve registration technique which consists of estimating a transformation which maps one time-basis (of the first curve) onto another time-basis (the second curve). We also analyse in depth the conditions under which such curve registration techniques are valid. Finally, we show results with two similar athletic events performed by two different athletes.

Image interpolation for virtual sports scenarios

View interpolation has been explored in the scientific community as a means to avoid the complexity of full 3D in the construction of photo-realistic interactive scenarios. EVENTS project attempts to apply state of the art view interpolation to the field of professional sports. The aim is to populate a wide scenario such as a stadium with a number of cameras and, via computer vision, to produce photo-realistic moving or static images from virtual viewpoints, i.e where there is no physical camera.EVENTS proposes an innovative view interpolation scheme based on the Joint View Triangulation algorithm developed by the project participants. Joint View Triangulation is combined within the EVENTS framework with new initiatives in the field of multiple view layered representation, automatic seed matching, image-based rendering, tracking occluding layers and constrained scene analysis. The computer vision software has been implemented on top of a novel high performance computing platform with the aim to achieve real-time interpolation.