Zhiqian Wang

Human activity recognition using multidimensional indexing

In this paper, we develop a novel method for view-based recognition of human action/activity from videos. By observing just a few frames, we can identify the activity that takes place in a video sequence. The basic idea of our method is that activities can be positively identified from a sparsely sampled sequence of a few body poses acquired from videos. In our approach, an activity is represented by a set of pose and velocity vectors for the major body parts (hands, legs, and torso) and stored in a set of multidimensional hash tables. We develop a theoretical foundation that shows that robust recognition of a sequence of body pose vectors can be achieved by a method of indexing and sequencing and it requires only a few pose vectors (i.e., sampled body poses in video frames). We find that the probability of false alarm drops exponentially with the increased number of sampled body poses. So, matching only a few body poses guarantees high probability for correct recognition. Our approach is parallel, i.e., all possible model activities are examined at one indexing operation. In addition, our method is robust to partial occlusion since each body part is indexed separately. We use a sequence-based voting approach to recognize the activity invariant to the activity speed.

Pictorial recognition of objects employing affine invariance in the frequency domain

Describes an efficient approach to pose invariant pictorial object recognition employing spectral signatures of image patches that correspond to object surfaces which are roughly planar. Based on singular value decomposition (SVD), the affine transform is decomposed into slant, tilt, swing, scale, and 2D translation. Unlike previous log-polar representations which were not invariant to slant, our log-log sampling configuration in the frequency domain yields complete affine invariance. The images are preprocessed by a novel model-based segmentation scheme that detects and segments objects that are affine-similar to members of a model set of basic geometric shapes. The segmented objects are then recognized by their signatures using multidimensional indexing in a pictorial dataset represented in the frequency domain. Experimental results with a dataset of 26 models show 100 percent recognition rates in a wide range of 3D pose parameters and imaging degradations: 0-360/spl deg/ swing and tilt, 0-82/spl deg/ of slant, more than three octaves in scale change, window-limited translation, high noise levels (0 dB), and significantly reduced resolution (1:5).

Optimal ramp edge detection using expansion matching

In practical images, ideal step edges are actually transformed into ramp edges, due to the general low pass filtering nature of imaging systems. This paper discusses the application of the expansion matching (EXM) method for optimal ramp edge detection. EXM optimizes a novel matching criterion called discriminative signal-to-noise ratio (DSNR) and has been shown to robustly recognize templates under conditions of noise, severe occlusion, and superposition. We show that our ramp edge detector performs better than the ramp detector obtained from Cannys criteria in terms of DSNR and is relatively easier to derive for various noise levels and slopes.

Iconic recognition with affine-invariant spectral signatures

This paper presents a new approach for object recognition using affine-invariant recognition of image patches that correspond to object surfaces that are roughly planar. A novel set of affine-invariant spectral signatures (AISSs) are used to recognize each surface separately invariant to its 3D pose. These local spectral signatures are extracted by correlating the image with a novel configuration of Gaussian kernels. The spectral signature of each image patch is then matched against a set of iconic models using multidimensional indexing (MDI) in the frequency domain. Affine-invariance of the signatures is achieved by a new configuration of Gaussian kernels with modulation in two orthogonal axes. The proposed configuration of kernels is Cartesian with varying aspect ratios in two orthogonal directions. The kernels are organized in subsets where each subset has a distinct orientation. Each subset spans the entire frequency domain and provides invariance to slant, scale and limited translation. The complete set of orientations is utilized to achieve invariance to rotation and tilt. Hence, the proposed set of kernels achieve complete affine-invariance.

Pictorial recognition using affine-invariant spectral signatures

This paper describes an efficient approach to pose invariant object recognition employing pictorial recognition of image patches. A complete affine invariance is achieved by a representation which is based on a new sampling configuration in the frequency domain. Employing Singular Value Decomposition (SVD), the affine transform is decomposed into slant, tilt, swing, scale and 2D translation. From this decomposition, we derive an affine invariant representation that allows to recognize image patches that correspond to object surfaces which are roughly planar-invariant to their pose in space. The representation is in the form of Spectral Signatures that are derived from a set of Cartesian logarithmic-logarithmic (log-log) sampling configuration in the frequency domain. Unlike previous log-polar representations which are not invariant to slant (i.e. foreshortening only in one direction), our new configuration yields complete affine invariance. The proposed log-log configuration can be employed both globally or locally by a Gabor or Fourier transforms. Local representation enables to recognize separately several objects in the same image. The actual signature recognition is performed by multidimensional indexing in a pictorial dataset represented in the frequency domain. The recognition also provides 3D pose information.

Affine invariant shape representation and recognition using Gaussian kernels and multi-dimensional indexing

This paper presents a new approach for object recognition using affine-invariant recognition of image patches that correspond to object surfaces that are roughly planar. A novel set of affine-invariant spectral signatures (AISSs) are used to recognize each surface separately invariant to its 3D pose. These local spectral signatures are extracted by convolving the image with a novel configuration of Gaussian kernels. The spectral signature of each image patch is then matched against a set of iconic models using multi-dimensional indexing (MDI) in the frequency domain. Affine-invariance of the signatures is achieved by a new configuration of Gaussian kernels with modulation in two orthogonal axes. The proposed configuration of kernels is Cartesian with varying aspect ratios in two orthogonal directions. The kernels are organized in subsets where each subset has a distinct orientation. Each subset spans the entire frequency domain and provides invariance to slant, scale and limited translation. The complete set of orientations is utilized to achieve invariance to rotation and tilt. Hence, the proposed set of kernels achieve complete affine-invariance.

SVD and log-log frequency sampling with Gabor kernels for invariant pictorial recognition

This paper presents an efficient scheme for affine-invariant object recognition. Affine invariance is obtained by a representation which is based on a new sampling configuration in the frequency domain. We discuss the decomposition of affine transform into slant, tilt, swing, scale and 2D translation by applying singular value decomposition (SVD). The affine invariant spectral signatures (AISS) are derived from a set of Cartesian logarithmic-logarithmic (log-log) sampling configuration in the frequency domain. The AISS enables the recognition of image patches that correspond to roughly planar object surfaces-regardless of their poses in space. Unlike previous log-polar representations which are not invariant to slant (i.e. foreshortening only in one direction), the AISS yields a complete affine invariance. The proposed log-log configuration can be employed either by a global Fourier transform or by a local Gabor transform. Local representation enables one to recognize separately several objects in the same image. The actual signature recognition is performed by multi-dimensional indexing in a pictorial dataset. 3D pose information is also derived as a by-product.

Generic object detection using model based segmentation

This paper presents a novel approach for detection and segmentation of generic shapes in cluttered images. The underlying assumption is that generic objects that are man made, frequently have surfaces which closely resemble standard model shapes such as rectangles, semi-circles etc. Due to the perspective transformations of optical imaging systems, a model shape may appear differently in the image with various orientations and aspect ratios. The set of possible appearances can be represented compactly by a few vectorial eigenbases that are derived from a small set of model shapes which are affine transformed in a wide parameter range. Instead of regular boundary of standard models, we apply a vectorial boundary which improves robustness to noise, background clutter and partial occlusion. The detection of generic shapes is realized by detecting local peaks of a similarity measure between the image edge map and an eigenspace combined set of the appearances. At each local maxima, a fast search approach based on a novel representation by an angle space is employed to determine the best matching between models and the underlying subimage. We find that angular representation in multidimensional search corresponds better to Euclidean distance than conventional projection and yields improved classification of noisy shapes. Experiments are performed in various interfering distortions, and robust detection and segmentation are achieved.

Model based segmentation and detection of affine transformed shapes in cluttered images

This paper presents a novel approach for detection and segmentation of generic shapes in cluttered images. The underlying assumption is that man made objects frequently have surfaces which closely resemble standard model shapes such as rectangles, semicircles etc. Due to the transformation of optical imaging systems, a model shape can appear differently in the image with different orientations and aspect ratios. This set of possible appearances can be represented compactly by a few vectorial eigenbases that are derived from a small set of model shapes which are affine transformed in a wide range. The use of vectorial boundary information improves robustness to noise, background clutter and partial occlusion. The detection of generic shapes is realized by detecting local peaks of a similarity measure between the image edge map and an eigenspace combined set of the appearances. At each local maxima, a fast search approach based on a novel representation by angle space is employed to determine the best matching between models and the underlying subimage. Experiments are performed in various interfering distortions, and robust detection and recognition are achieved.

Robust shape description and recognition by gradient propagation

This paper presents a novel hierarchical shape description scheme based on propagating the gradient of the image. The propagated gradient field collides at centers of convex/concave shape components, which can be detected as points of high directional disparity. A novel vectorial disparity measure called cancelation energy is used to measure this collision of the gradient field, and local maxima of this measure yield feature tokens. These feature tokens form a compact description of shapes and their components and indicate their central location and size. In addition, a gradient signature is formed by the gradient field that collides at each center, which is itself a robust and size-independent description of the corresponding shape component. Experimental results demonstrate that the shape description is robust to distortion, noise and clutter. An important advantage of this scheme is that the feature tokens are obtained pre-attentively, without prior understanding of the image. The hierarchical description is also successfully used for similarity-invariant recognition of 2D shapes with a multi-dimensional indexing scheme based on the gradient signature.