Edward Hunter

Human Body Model Acquisition and Tracking Using Voxel Data

We present an integrated system for automatic acquisition of the human body model and motion tracking using input from multiple synchronized video streams. The video frames are segmented and the 3D voxel reconstructions of the human body shape in each frame are computed from the foreground silhouettes. These reconstructions are then used as input to the model acquisition and tracking algorithms.The human body model consists of ellipsoids and cylinders and is described using the twists framework resulting in a non-redundant set of model parameters. Model acquisition starts with a simple body part localization procedure based on template fitting and growing, which uses prior knowledge of average body part shapes and dimensions. The initial model is then refined using a Bayesian network that imposes human body proportions onto the body part size estimates. The tracker is an extended Kalman filter that estimates model parameters based on the measurements made on the labeled voxel data. A voxel labeling procedure that handles large frame-to-frame displacements was designed resulting in very robust tracking performance.Extensive evaluation shows that the system performs very reliably on sequences that include different types of motion such as walking, sitting, dancing, running and jumping and people of very different body sizes, from a nine year old girl to a tall adult male.

Automated diagnosis and image understanding with object extraction, object classification, and inferencing in retinal images

Medical imaging is shifting from film to electronic images. The STARE (structured analysis of the retina) system is a sophisticated image management system that will automatically diagnose images, compare images, measure key features in images, annotate image contents, and search for images similar in content. The authors concentrate on automated diagnosis. The images are annotated by segmentation of objects of interest, classification of the extracted objects, and reasoning about the image contents. The inferencing is accomplished with Bayesian networks that learn from image examples of each disease. This effort at image understanding in fundus images anticipates the future use of medical images. As these capabilities mature, the authors expect that ophthalmologists and physicians in other fields that rely in images will use a system like STARE to reduce repetitive work, to provide assistance to physicians in difficult diagnoses or with unfamiliar diseases, and to manage images in large image databases.

Articulated body posture estimation from multi-camera voxel data

We present a framework for articulated body model acquisition and tracking from voxel data. A 3D voxel reconstruction of the persons body is computed from silhouettes extracted from four cameras. The model acquisition process is fully automated. In the first frame, body parts are located sequentially. The head is located first, since its shape and size are unique and stable. Other parts are found by sequential template growing and fitting. This initial estimate of body part locations, sizes and orientations is then used as a measurement for the extended Kalman filter which ensures a valid articulated body model. The same filter, with a slightly modified state and state transition matrix, is then used for tracking. The performance of the system has been evaluated on several video sequences with promising results.

Recursive identification of gesture inputs using hidden Markov models

Human-machine interfaces play a role of growing importance as computer technology continues to evolve. Motivated by the desire to provide users with an intuitive gesture input system, we describe the design of a recursive filter applied to the vision-based gesture interpretation problem. The gestures are modeled as a hidden Markov model with the state representing the gesture sequences, and the observations being the current static hand pose. At each time step the recursive filter updates its estimate of what gesture is occurring based on the current extracted pose information. The result is a robust system which provides the user with continual feedback during compound gestures.<<ETX>>

Advances in molecular labeling, high throughput imaging and machine intelligence portend powerful functional cellular biochemistry tools

Cellular behavior is complex. Successfully understanding systems at ever‐increasing complexity is fundamental to advances in modern science and unraveling the functional details of cellular behavior is no exception. We present a collection of prospectives to provide a glimpse of the techniques that will aid in collecting, managing and utilizing information on complex cellular processes via molecular imaging tools. These include: 1) visualizing intracellular protein activity with fluorescent markers, 2) high throughput (and automated) imaging of multilabeled cells in statistically significant numbers, and 3) machine intelligence to analyze subcellular image localization and pattern. Although not addressed here, the importance of combining cell‐image‐based information with detailed molecular structure and ligand‐receptor binding models cannot be overlooked. Advanced molecular imaging techniques have the potential to impact cellular diagnostics for cancer screening, clinical correlations of tissue molecular patterns for cancer biology, and cellular molecular interactions for accelerating drug discovery. The goal of finally understanding all cellular components and behaviors will be achieved by advances in both instrumentation engineering (software and hardware) and molecular biochemistry. J. Cell. Biochem. Suppl. 39: 194–210, 2002.

Vision based hand gesture interpretation using recursive estimation

Gesture recognition requires spatio-temporal image sequence analysis. The actual length of the sequence varies with each instantiation of the gesture, and can be quite long in the case of a multiple gesture sequence. To achieve adequate system response we introduce the concept of recursive estimation of the gesture state. This consists of modeling the gestures as a sequence of static hand poses. Using a hidden Markov model where the unobservable state is the spatio-temporal gesture and the hand poses are the observations allows us to determine the current probabilities of each gesture with a finite state estimator. This decomposes the gesture recognition process into two stages: identification of the hand pose within the current image frame and incorporation of the new information into the probability estimates. We illustrate the performance of the estimator by describing the implementation of a telerobotic application.<<ETX>>

MPI-Video infrastructure for dynamic environments

We describe an infrastructure for multiple perspective interactive video (MPI-Video) systems. An MPI-Video system assimilates data from multiple disparate sensors, allowing a user to interact with them in a meaningful way. At the heart of an MPI-Video system is an environment model (EM) that both assimilates data from multiple sensors and acts as an information server to clients. A collection of strongly-coupled dynamic state estimation algorithms does the assimilation. Results show that our infrastructure is easy to configure, easy to extend, and is effective at modeling a dynamic environment.

Accuracy of least squares designed spatial FIR filters for segmentation of images of fluorescence stained cell nuclei.

A method for accurate, real-time image segmentation is needed for the development of a fully automated image cytometer that combines the speed and case-of-use of flow cytometry with the detailed morphometry of imaging. Object intensity variation and inherent optical blur make real-time segmentation challenging. The best spatial finite impulse response (FIR) filter, implemented as a convolution, was tested for sharpening edges and creating the required contrast. The filter and threshold segmentation steps were treated as a two-category linear classifier. Best 3 x 3 through 25 x 25 filters were designed utilizing the perceptron criterion and nonlinear least squares, and tested on ten montage images of a combined 1,070 manually segmented DAPI stained cell nuclei. The resulting image contrast, or class separation, led to simple automatic thresholding via the histogram intermodal minimum. Image segmentation accuracy began to plateau at 7 x 7 filters and did not increase above 15 x 15. Little loss in accuracy occurred with application to the images not used for design. This segmentation method provides a systematic, fast and accurate means of creating binary object maps useful for subsequent measurement, processing and cell classification.

Estimation of articulated motion using kinematically constrained mixture densities

We address the problem of articulated posture estimation in its general form. Namely, the recovery of full 3D articulated posture parameters from an uncontrolled scene. Stochastic modeling of low-level segmented image data is unified with models of object kinematic structure through a constrained mixture of observation processes. A modified expectation-maximization algorithm is proposed for this purpose. Early experiments qualitatively demonstrate the efficacy of our approach, and provide a context for integration for more sophisticated image cues.

A Live Cell, Image‐Based Approach to Understanding the Enzymology and Pharmacology of 2‐Bromopalmitate and Palmitoylation

The addition of a lipid moiety to a protein increases its hydrophobicity and subsequently its attraction to lipophilic environments like membranes. Indeed most lipid-modified proteins are localized to membranes where they associate with multiprotein signaling complexes. Acylation and prenylation are the two common categories of lipidation. The enzymology and pharmacology of prenylation are well understood but relatively very little is known about palmitoylation, the most common form of acylation. One distinguishing characteristic of palmitoylation is that it is a dynamic modification. To understand more about how palmitoylation is regulated, we fused palmitoylation substrates to fluorescent proteins and reported their subcellular distribution and trafficking. We used automated high-throughput fluorescence microscopy and a specialized computer algorithm to image and measure the fraction of palmitoylation reporter on the plasma membrane versus the cytoplasm. Using this system we determined the residence half-life of palmitate on the dipalmitoyl substrate peptide from GAP43 as well as the EC(50) for 2-bromopalmitate, a common inhibitor of palmitoylation.