Jezekiel Ben-Arie

Human activity recognition using multidimensional indexing

In this paper, we develop a novel method for view-based recognition of human action/activity from videos. By observing just a few frames, we can identify the activity that takes place in a video sequence. The basic idea of our method is that activities can be positively identified from a sparsely sampled sequence of a few body poses acquired from videos. In our approach, an activity is represented by a set of pose and velocity vectors for the major body parts (hands, legs, and torso) and stored in a set of multidimensional hash tables. We develop a theoretical foundation that shows that robust recognition of a sequence of body pose vectors can be achieved by a method of indexing and sequencing and it requires only a few pose vectors (i.e., sampled body poses in video frames). We find that the probability of false alarm drops exponentially with the increased number of sampled body poses. So, matching only a few body poses guarantees high probability for correct recognition. Our approach is parallel, i.e., all possible model activities are examined at one indexing operation. In addition, our method is robust to partial occlusion since each body part is indexed separately. We use a sequence-based voting approach to recognize the activity invariant to the activity speed.

The probabilistic peaking effect of viewed angles and distances with application to 3-D object recognition

Two novel probabilistic models for viewed angles and distances are derived using an observability sphere method. The method, which is based on the assumption that the prior probability density is isotropic for all viewing orientations, can be used for the computation of observation probabilities for objects aspects, features, and probability densities of their quantitative attributes. Using the sphere, it is discovered that the probability densities of viewed angles, distances, and even projected curvature have sharp peaks at their original values. From this peaking effect, it is concluded that in most cases, the values of angles and distances are being altered only slightly by the imaging process, and they can still serve as a strong cue for model-based recognition. The probabilistic models for 3-D object recognition from monocular images are used. To form the angular elements that are needed, the objects are represented by their linear features and specific points primitives. Using the joint density model of angles and distances, the probabilities of initial matching hypotheses and mutual information coefficients are estimated. These results are then used for object recognition by optimal matching search and stochastic labeling schemes. Various synthetic and real objects are recognized by this approach. >

Pictorial recognition of objects employing affine invariance in the frequency domain

Describes an efficient approach to pose invariant pictorial object recognition employing spectral signatures of image patches that correspond to object surfaces which are roughly planar. Based on singular value decomposition (SVD), the affine transform is decomposed into slant, tilt, swing, scale, and 2D translation. Unlike previous log-polar representations which were not invariant to slant, our log-log sampling configuration in the frequency domain yields complete affine invariance. The images are preprocessed by a novel model-based segmentation scheme that detects and segments objects that are affine-similar to members of a model set of basic geometric shapes. The segmented objects are then recognized by their signatures using multidimensional indexing in a pictorial dataset represented in the frequency domain. Experimental results with a dataset of 26 models show 100 percent recognition rates in a wide range of 3D pose parameters and imaging degradations: 0-360/spl deg/ swing and tilt, 0-82/spl deg/ of slant, more than three octaves in scale change, window-limited translation, high noise levels (0 dB), and significantly reduced resolution (1:5).

Optimal ramp edge detection using expansion matching

In practical images, ideal step edges are actually transformed into ramp edges, due to the general low pass filtering nature of imaging systems. This paper discusses the application of the expansion matching (EXM) method for optimal ramp edge detection. EXM optimizes a novel matching criterion called discriminative signal-to-noise ratio (DSNR) and has been shown to robustly recognize templates under conditions of noise, severe occlusion, and superposition. We show that our ramp edge detector performs better than the ramp detector obtained from Cannys criteria in terms of DSNR and is relatively easier to derive for various noise levels and slopes.

A volumetric/iconic frequency domain representation for objects with application for pose invariant face recognition

A novel method for representing 3D objects that unifies viewer and model centered object representations is presented. A unified 3D frequency-domain representation, called volumetric frequency representation (VFR), encapsulates both the spatial structure of the object and a continuum of its views in the same data structure. The frequency-domain image of an object viewed from any direction can be directly extracted employing an extension of the projection slice theorem, where each Fourier-transformed view is a planar slice of the volumetric frequency representation. The VFR is employed for pose-invariant recognition of complex objects, such as faces. The recognition and pose estimation is based on an efficient matching algorithm in a four-dimensional Fourier space. Experimental examples of pose estimation and recognition of faces in various poses are also presented.

Optimal template matching by nonorthogonal image expansion using restoration

In this paper we present a novel approach for template matching. The basic principle is expansion matching and it entails signal expansion into a set of nonorthogonal templatesimilar basis functions. The coefficients of this expansion signify the presence of the template in the corresponding locations in the image. We demonstrate that this matching technique is robust in conditions of noise, superposition, and severe occlusion. A new and more practical discriminative signal-to-noise ratio (DSNR) for matching is proposed that considers even the filters off-center response to the template as “noise”. We show that expansion yields the optimal linear operator that maximizes the DSNR and results in a sharp response to the matched template. Theoretical and experimental comparisons of expansion matching and the widely used correlation matching demonstrate the superiority of our approach. Correlation matching (also known as matched filtering) yields broad peaks and spurious responses, both of which hamper good detection. We also show that the special case of expansion with a dense set of self-similar basis functions is equivalent to signal restoration. Expansion matching can be implemented by restoration techniques and also by our recently developed lattice architecture.

Minimal classification method with error-correcting codes for multiclass recognition

In this work, we develop an efficient technique to transform a multiclass recognition problem into a minimal binary classification problem using the Minimal Classification Method (MCM). The MCM requires only log2 N classifications whereas the other methods require much more. For the classification, we use Support Vector Machine (SVM) based binary classifiers since they have superior generalization performance. Unlike the prevalent one-versus-one strategy (the bottom-up one-versus-one strategy is called tournament method) that separates only two classes at each classification, the binary classifiers in our method have to separate two groups of multiple classes. As a result, the probability of generalization error increases. This problem is alleviated by utilizing error correcting codes, which results only in a marginal increase in the required number of classifications. However, in comparison to the tournament method, our method requires only 50% of the classifications and still similar performance can be attained. The proposed solution is tested with the Columbia Object Image Library (COIL). We also test the performance under conditions of noise and occlusion.

Signal representation by generalized nonorthogonal Gaussian wavelet groups using lattice networks

The authors describe a general method for signal representation using nonorthogonal basis functions that are composed of Gaussians. The Gaussians can be combined into groups with predetermined configuration that can approximate any desired basis function. The same configuration at different scales forms a set of self-similar wavelets. The general scheme is demonstrated by representing a natural signal using an arbitrary basis function. The basic methodology is demonstrated by two novel schemes for efficient representation of 1-D and 2-D signals using Gaussian basis functions (BFs). Special methods are required here since the Gaussian functions are nonorthogonal. The first method uses a paradigm of maximum energy reduction interlaced with the A* heuristic search. The second method uses an adaptive lattice system to find the minimum-squared error of the BFs onto the signal, and a lateral-vertical suppression network to select the most efficient representation in terms of data compression.<<ETX>>

A neural network approach for reconstructing surface shape from shading

A framework for the reconstruction of smooth surface shapes from shading images is presented. The method is based on using a backpropagation based neural network for learning brightness patterns and associating them with range data. The network is designed to reconstruct surface range from localized intensity patches of 7/spl times/7 pixels. Two methods for training the network are investigated, one based on a novel weight diffusion process which enforces a local smoothness constraint and the other using the eigen coefficients of the input and output patterns which make the training computationally efficient. An elegant and simple method for integrating reconstructed surface patches by minimizing the sum squared error in overlapped areas is derived. Results are shown for reconstruction of simple shapes like cylinders, hyperboloids and paraboloids as well as complex shapes like facial structure from intensity images.

Iconic recognition with affine-invariant spectral signatures

This paper presents a new approach for object recognition using affine-invariant recognition of image patches that correspond to object surfaces that are roughly planar. A novel set of affine-invariant spectral signatures (AISSs) are used to recognize each surface separately invariant to its 3D pose. These local spectral signatures are extracted by correlating the image with a novel configuration of Gaussian kernels. The spectral signature of each image patch is then matched against a set of iconic models using multidimensional indexing (MDI) in the frequency domain. Affine-invariance of the signatures is achieved by a new configuration of Gaussian kernels with modulation in two orthogonal axes. The proposed configuration of kernels is Cartesian with varying aspect ratios in two orthogonal directions. The kernels are organized in subsets where each subset has a distinct orientation. Each subset spans the entire frequency domain and provides invariance to slant, scale and limited translation. The complete set of orientations is utilized to achieve invariance to rotation and tilt. Hence, the proposed set of kernels achieve complete affine-invariance.