W.E.L. Grimson

Learning patterns of activity using real-time tracking

Our goal is to develop a visual monitoring system that passively observes moving objects in a site and learns patterns of activity from those observations. For extended sites, the system will require multiple cameras. Thus, key elements of the system are motion tracking, camera coordination, activity classification, and event detection. In this paper, we focus on motion tracking and show how one can use observed motion to learn patterns of activity in a site. Motion segmentation is based on an adaptive background subtraction method that models each pixel as a mixture of Gaussians and uses an online approximation to update the model. The Gaussian distributions are then evaluated to determine which are most likely to result from a background process. This yields a stable, real-time outdoor tracker that reliably deals with lighting changes, repetitive motions from clutter, and long-term scene changes. While a tracking system is unaware of the identity of any object it tracks, the identity remains the same for the entire tracking sequence. Our system leverages this information by accumulating joint co-occurrences of the representations within a sequence. These joint co-occurrence statistics are then used to create a hierarchical binary-tree classification of the representations. This method is useful for classifying sequences, as well as individual instances of activities in a site.

Adaptive segmentation of MRI data

Intensity-based classification of MR images has proven problematic, even when advanced techniques are used. Intra-scan and inter-scan intensity inhomogeneities are a common source of difficulty. While reported methods have had some success in correcting intra-scan inhomogeneities, such methods require supervision for the individual scan. This paper describes a new method called adaptive segmentation that uses knowledge of tissue intensity properties and intensity inhomogeneities to correct and segment MR images. Use of the EM algorithm leads to a fully automatic method that allows for more accurate segmentation of tissue types as well as better visualization of MRI data, that has proven to be effective in a study that includes more than 1000 brain scans.

Statistical shape influence in geodesic active contours

A novel method of incorporating shape information into the image segmentation process is presented. We introduce a representation for deformable shapes and define a probability distribution over the variances of a set of training shapes. The segmentation process embeds an initial curve as the zero level set of a higher dimensional surface, and evolves the surface such that the zero level set converges on the boundary of the object to be segmented. At each step of the surface evolution, we estimate the maximum a posteriori (MAP) position and shape of the object in the image, based on the prior shape information and the image information. We then evolve the surface globally, towards the MAP estimate, and locally, based on image gradients and curvature. Results are demonstrated on synthetic data and medical imagery, in 2D and 3D.

A shape-based approach to the segmentation of medical imagery using level sets

We propose a shape-based approach to curve evolution for the segmentation of medical images containing known object types. In particular, motivated by the work of Leventon, Grimson, and Faugeras (2000), we derive a parametric model for an implicit representation of the segmenting curve by applying principal component analysis to a collection of signed distance representations of the training data. The parameters of this representation are then manipulated to minimize an objective function for segmentation. The resulting algorithm is able to handle multidimensional data, can deal with topological changes of the curve, is robust to noise and initial contour placements, and is computationally efficient. At the same time, it avoids the need for point correspondences during the training phase of the algorithm. We demonstrate this technique by applying it to two medical applications; two-dimensional segmentation of cardiac magnetic resonance imaging (MRI) and three-dimensional segmentation of prostate MRI.

Using adaptive tracking to classify and monitor activities in a site

We describe a vision system that monitors activity in a site over extended periods of time. The system uses a distributed set of sensors to cover the site, and an adaptive tracker detects multiple moving objects in the sensors. Our hypothesis is that motion tracking is sufficient to support a range of computations about site activities. We demonstrate using the tracked motion data to calibrate the distributed sensors, to construct rough site models, to classify detected objects, to learn common patterns of activity for different object classes, and to detect unusual activities.

An automatic registration method for frameless stereotaxy, image guided surgery, and enhanced reality visualization

There is a need for frameless guidance systems to help surgeons plan the exact location for incisions, to define the margins of tumors, and to precisely identify locations of neighboring critical structures. The authors have developed an automatic technique for registering clinical data, such as segmented magnetic resonance imaging (MRI) or computed tomography (CT) reconstructions, with any view of the patient on the operating table. The authors demonstrate on the specific example of neurosurgery. The method enables a visual mix of live video of the patient and the segmented three-dimensional (3-D) MRI or CT model. This supports enhanced reality techniques for planning and guiding neurosurgical procedures and allows us to interactively view extracranial or intracranial structures nonintrusively. Extensions of the method include image guided biopsies, focused therapeutic procedures, and clinical studies involving change detection over time sequences of images.

On the sensitivity of the Hough transform for object recognition

Object recognition from sensory data involves, in part, determining the pose of a model with respect to a scene. A common method for finding an objects pose is the generalized Hough transform, which accumulates evidence for possible coordinate transformations in a parameter space whose axes are the quantized transformation parameters. Large clusters of similar transformations in that space are taken as evidence of a correct match. A theoretical analysis of the behavior of such methods is presented. The authors derive bounds on the set of transformations consistent with each pairing of data and model features, in the presence of noise and occlusion in the image. Bounds are provided on the likelihood of false peaks in the parameter space, as a function of noise, occlusion, and tessellation effects. It is argued that haphazardly applying such methods to complex recognition tasks is risky, as the probability of false positives can be very high. >

CURVES: Curve evolution for vessel segmentation

The vasculature is of utmost importance in neurosurgery. Direct visualization of images acquired with current imaging modalities, however, cannot provide a spatial representation of small vessels. These vessels, and their branches which show considerable variations, are most important in planning and performing neurosurgical procedures. In planning they provide information on where the lesion draws its blood supply and where it drains. During surgery the vessels serve as landmarks and guidelines to the lesion. The more minute the information is, the more precise the navigation and localization of computer guided procedures. Beyond neurosurgery and neurological study, vascular information is also crucial in cardiovascular surgery, diagnosis, and research. This paper addresses the problem of automatic segmentation of complicated curvilinear structures in three-dimensional imagery, with the primary application of segmenting vasculature in magnetic resonance angiography (MRA) images. The method presented is based on recent curve and surface evolution work in the computer vision community which models the object boundary as a manifold that evolves iteratively to minimize an energy criterion. This energy criterion is based both on intensity values in the image and on local smoothness properties of the object boundary, which is the vessel wall in this application. In particular, the method handles curves evolving in 3D, in contrast with previous work that has dealt with curves in 2D and surfaces in 3D. Results are presented on cerebral and aortic MRA data as well as lung computed tomography (CT) data.

An implementation of a computational theory of visual surface interpolation

Abstract Computational theories of structure-from-motion ( Ullman, The Interpretation of Visual Motion , MIT Press, 1979 ) and stereo vision ( Marr and Poggio, Proc. R. Soc. London Ser. B 204 , 1979, 301–328 ) only specify the computation of three-dimensional surface information at particular points in the image. Yet, the visual perception is clearly of complete surfaces. To account for this, a computational theory of the interpolation of surfaces from visual information was presented in Grimson. ( From Images to Surfaces: A Computational Study of the Human Early Visual System , MIT Press, 1981 ; A Computational Theory of Visual Surface Interpolation , MIT Artificial Intelligence Lab Memo, No. 613, 1981 ; and Philos. Trans. R. Soc. London Ser. B 298 , 1982, 395–427 ). The problem is constrained by the fact that the surface must agree with the information from stereo or motion correspondence, and not vary radically between these points. Using the image irradiance equation ( Horn, MIT Project MAC Tech. Rep. MACTR-79, 1970 ; The Psychology of Computer Vision , McGraw-Hill, 1975 ; and Artif. Intell. 8 , 1977, 201–231 ), an explicit form of this surface consistency constraint can be derived ( Grimson, MIT Artificial Intelligence Lab Memo, No. 646, 1981 ). To determine which of two possible surfaces is more consistent with the surface consistency constraint, one must be able to compare the two surfaces. To do this, a functional from the space of possible functions to the real numbers is required. In this way, the surface most consistent with the visual information will be that which minimizes the functional. In Grimson, a set of conditions was derived which ensures that the functional has a unique minimal surface. Based on these conditions, a number of possible functionals were proposed. In Brady and Horn (MIT Artificial Intelligence Lab Memo, No. 654, 1981 ), it was shown that this set of possible functionals forms a vector space, spanned by the functional of quadratic variation and the functional of the square Laplacian. Analytic arguments were given in Grimson to support the choice of the quadratic variation as the functional whose minimal surface is the “best” interpolation of the known points. In this paper, algorithms for computing the minimal surface are derived. Using this implementation of the computational theory derived in Grimson the differences between minimal surfaces computed using quadratic variation and those computed using the square Laplacian are illustrated. These examples provide additional support for the choice of the quadratic variation. The performance of the algorithm in interpolating both random dot and natural stereograms which have been processed by the Marr-Poggio stereo algorithm ( Grimson, Computing Shape Using a Theory of Human Stereo Vision, Ph.D. Thesis, MIT, Cambridge, 1980 and Philos. Trans. R. Soc. London Ser. B 292 , 1981, 217–253 ) is also illustrated.

Edge-based rich representation for vehicle classification

In this paper, we propose an approach to vehicle classification under a mid-field surveillance framework. We develop a repeatable and discriminative feature based on edge points and modified SIFT descriptors, and introduce a rich representation for object classes. Experimental results show the proposed approach is promising for vehicle classification in surveillance videos despite great challenges such as limited image size and quality and large intra-class variations. Comparisons demonstrate the proposed approach outperforms other methods