Kuntal Sengupta

Organizing large structural modelbases

Presents a hierarchically structured approach to organizing large structural modelbases using an information theoretic criterion. Objects (patterns) are modeled in the form of random parametric structural descriptions (RPSDs), an extension of the parametric structural description graph-theoretic formalism. Objects in scenes are modeled as parametric structural descriptions (PSDs). The organization process is driven by pair-wise dissimilarity values between RPSDs. The authors also introduce the node pointer lists, which are computed offline during modelbase organization. During recognition, the only exponential matching process involved is between the scene PSD and the RPSD at the root of the organized tree. Using the organized hierarchy along with the node pointer lists, the remaining work simplifies to a series of inexpensive linear tests at the subsequent levels of the tree search. The authors develop the theory and present three modelbases: 50 objects built from real image data, 100 CAD models, and 1000 synthetic abstract models. >

Cooperative Multitarget Tracking With Efficient Split and Merge Handling

For applications such as behavior recognition it is important to maintain the identity of multiple targets, while tracking them in the presence of splits and merges, or occlusion of the targets by background obstacles. Here we propose an algorithm to handle multiple splits and merges of objects based on dynamic programming and a new geometric shape matching measure. We then cooperatively combine Kalman filter-based motion and shape tracking with the efficient and novel geometric shape matching algorithm. The system is fully automatic and requires no manual input of any kind for initialization of tracking. The target track initialization problem is formulated as computation of shortest paths in a directed and attributed graph using Dijkstras shortest path algorithm. This scheme correctly initializes multiple target tracks for tracking even in the presence of clutter and segmentation errors which may occur in detecting a target. We present results on a large number of real world image sequences, where upto 17 objects have been tracked simultaneously in real-time, despite clutter, splits, and merges in measurements of objects. The complete tracking system including segmentation of moving objects works at 25 Hz on 352times288 pixel color image sequences on a 2.8-GHz Pentium-4 workstation

Modelbase Partitioning Using Property Matrix Spectra

In this paper, we present aneigenvalueorspectral-basedrepresentation for CAD models to be used in conjunction with the more traditional attributed graph based representation of these models. The eigenvalues provide a gross description of the structure of the objects and help to divide a large modelbase into structurally homogeneous partitions. Models in each partition are next hierarchically organized according to the algorithm we presented in a previous paper IEEE Trans. Pattern Anal. Machine Intell.17, 1995, 321--332]. In recognition, gross features computed from an object in a range image are used to prune the modelbase by selecting a few “favorable” partitions in which the correct object model is likely to lie. We also model the perturbations in the eigenvalues computed from objects in real scenes and show how this perturbation model can be used effectively during recognition. The partitioning experiments presented here are for real range images using a modelbase of 125 CAD objects with planar, cylindrical, and spherical surfaces. From our recognition results, we observe that for a reasonable degree of error in the low-level processes (surface segmentation and grouping), the correct partition is always included. Experimental results also point to a significant increase in recognition speed, even on modelbases much smaller than the ones we consider here.

Information theoretic clustering of large structural modelbases

A hierarchically structured approach to organizing large structural model bases using an information theoretic criterion is presented. Objects (patterns) are modeled in the form of random parametric structural descriptions (RPSDs), an extension of the parametric structural description graph-theoretic formalism. Hierarchically clustering the RPSDs reduces the computational work to O(log N). The node pointers allow a mapping between the observation and a stored representation at one level, and the mapping to all potential models at all subsequent levels is reduced to mere tests, eliminating the exponential search for the best interprimitive mapping function for each stored candidate pattern.<<ETX>>

Using geometric hashing with information theoretic clustering for fast recognition from a large CAD modelbase

We introduce a geometric hashing strategy to recognize CAD models from an organized hierarchy. Unlike most prior work in hashing using graph theoretic models, this work is a step closer to the classical, point based geometric hashing scheme. The geometric hashing strategy is used along with the hierarchical organization strategy defined by K. Sengupta and K.L. Boyer (1995). The combination of these two concepts can potentially reduce the recognition time considerably, especially versus the normal graph theoretic ideas, while retaining all of their benefits. We also present an error analysis of the hashing scheme considering the sensor noise and the scene clutter. Experiments with a CAD modelbase and both synthetic and real images indicate the potential of this scheme for fast recognition from large modelbases.

Occlusion handling in augmented reality using background-foreground segmentation and projective geometry

Developing a seamless merging of real and virtual image streams and 3D models is an active research topic in augmented reality (AR). We propose a method for real-time augmentation of real videos with 2D and 3D objects by addressing the occlusion issue in an unique fashion. For virtual planar objects (such as images), the 2D overlay is automatically overlaid in a planar region selected by the user in the video. The overlay is robust to arbitrary camera motion. Furthermore, a unique background-foreground segmentation algorithm renders this augmented overlay as part of the background if it coincides with foreground objects in the video stream, giving the impression that it is occluded by foreground objects. The proposed technique does not require multiple cameras, camera calibration, use of fiducials, or a structural model of the scene to work. Extending the work further, we propose a novel method of augmentation by using trifocal tensors to augment 3D objects in 3D scenes to similar effect and implement it in real time as a proof of concept. We show several results of the successful working of our algorithm in real-life situations. The technique works on a real-time video from a USB camera, Creative Webcam III, onaPIV1.6GHz system without any special hardware support.

A non-parametric approach for independent component analysis using kernel density estimation

Learning using independent component analysis (ICA) has found a wide range of applications in the area of computer vision and pattern analysis, ranging from face recognition to speech separation. This paper presents a non-parametric approach to the ICA problem that is robust towards outlier effects. The algorithm, for the first time in the field of ICA, adopts an intuitive and direct approach, focusing on the very definition of independence itself; i.e. the joint probability density function (pdf) of independent sources is factorial over the marginal distributions. In the proposed algorithm, kernel density estimation is employed to approximate the underlying distributions. There are two major advantages of our algorithm. First, existing algorithms focus on learning the independent components by attempting to fulfill necessary conditions (but not sufficient) for independence. For example, the Jade algorithm attempts to approximate independence by minimizing higher order statistics, which are not robust to outliers. Comparatively, our technique is inherently robust towards outlier effects. Second, since the learning employs kernel density estimation, it is naturally free from the assumptions of source distributions (unlike the Infomax algorithm). Experimental results show that the algorithm is able to perform separation of sources in the presence of outliers, whereas existing algorithms like Jade and Infomax break down under such conditions. The results have also shown that the proposed non-parametric approach is generally source distribution independent. In addition, it is able to separate non-Gaussian zero-kurtotic signals unlike the traditional ICA algorithms like Jade and Infomax.

Experiments in semiautonomous registration of disparate spatial data: two new techniques and a model for comparison

We present two new methods for the semiautonomous registration of spatial data from disparate sources, in particular image and elevation data. One method stems from the synthetic image idea first proposed by Horn and Bachman, while the second is a line alignment technique. The methods estimate the affine transform between digital terrain elevation data (DTED) and aerial photographs from the National High Altitude Photography (NHAP) program. We also propose a novel measure of transform consistency and demonstrate the quality of the results. Both systems are accessed via graphical interfaces which are not described here.

Generating virtual environments for human communications: Virtual metamorphosis system and novel view generation

At ATR Media Integration & Communications Research Laboratories, our research group is aiming at realizing virtual communication environments in which the users at remote sites can feel that they are virtually co-located. This paper presents the ongoing research in two subareas chosen to address this problem: (a) Generating human images in virtual scenes. (b) An image based approach for generation of (background) scenes. In the virtual metamorphosis system, 3D models of characters are created in advance. For metamorphosis, the facial expressions and body posture of a person are detected in real-time from the face images and monocular thermal images of the person, respectively. The detected expressions and body movements are reproduced by deforming the 3D models. Demonstrations using 3D models of a Kabuki actor and a dinosaur show good performance. For novel view generation, we use as affine coordinate based reprojection scheme, which is also extended to merging real and synthetic objects.

A new camera projection model and its application in reprojection

We present a new camera projection model, which is intermediate between the affine camera model and the pin hole projection model. It is modeled as a perspective projection of 3D points into an arbitrary plane, followed by an affine transform of these projected points. We observe that the reprojection of a point into a novel image can be achieved uniquely provided that we have located a set of five reference points over four images (of which three are input images, and the fourth is the novel image). Also, the reprojection theory does not assume that the input images are captured from cameras with identical internal calibration parameters. Thus, we apply our technique to two different domains: 1) generation of novel images from a stereo pair; and 2) generation of virtual walk-through sequence with a monocular image sequence as input.