Ivana Mikic

Human Body Model Acquisition and Tracking Using Voxel Data

We present an integrated system for automatic acquisition of the human body model and motion tracking using input from multiple synchronized video streams. The video frames are segmented and the 3D voxel reconstructions of the human body shape in each frame are computed from the foreground silhouettes. These reconstructions are then used as input to the model acquisition and tracking algorithms.The human body model consists of ellipsoids and cylinders and is described using the twists framework resulting in a non-redundant set of model parameters. Model acquisition starts with a simple body part localization procedure based on template fitting and growing, which uses prior knowledge of average body part shapes and dimensions. The initial model is then refined using a Bayesian network that imposes human body proportions onto the body part size estimates. The tracker is an extended Kalman filter that estimates model parameters based on the measurements made on the labeled voxel data. A voxel labeling procedure that handles large frame-to-frame displacements was designed resulting in very robust tracking performance.Extensive evaluation shows that the system performs very reliably on sequences that include different types of motion such as walking, sitting, dancing, running and jumping and people of very different body sizes, from a nine year old girl to a tall adult male.

Segmentation and tracking in echocardiographic sequences: active contours guided by optical flow estimates

This paper presents a method for segmentation and tracking of cardiac structures in ultrasound image sequences. The developed algorithm is based on the active contour framework. This approach requires initial placement of the contour close to the desired position in the image, usually an object outline. Best contour shape and position are then calculated, assuming that at this configuration a global energy function, associated with a contour, attains its minimum. Active contours can be used for tracking by selecting a solution from a previous frame as an initial position in a present frame. Such an approach, however, fails for large displacements of the object of interest. This paper presents a technique that incorporates the information on pixel velocities (optical flow) into the estimate of initial contour to enable tracking of fast-moving objects. The algorithm was tested on several ultrasound image sequences, each covering one complete cardiac cycle. The contour successfully tracked boundaries of mitral valve leaflets, aortic root and endocardial borders of the left ventricle. The algorithm-generated outlines were compared against manual tracings by expert physicians. The automated method resulted in contours that were within the boundaries of intraobserver variability.

Moving shadow and object detection in traffic scenes

We present an algorithm for segmentation of traffic scenes that distinguishes moving objects from their moving cast shadows. A fading memory estimator calculates mean and variance of all three color components for each background pixel. Given the statistics for a background pixel, simple rules for calculating its statistics when covered by a shadow are used. Then, MAP classification decisions are made for each pixel. In addition to the color features, we examine the use of neighborhood information to produce smoother classification. We also propose the use of temporal information by modifying class a priori probabilities based on predictions from the previous frame.

Dynamic context capture and distributed video arrays for intelligent spaces

Intelligent environments can be viewed as systems where humans and machines (rooms) collaborate. Intelligent (or smart) environments need to extract and maintain an awareness of a wide range of events and human activities occurring in these spaces. This requirement is crucial for supporting efficient and effective interactions among humans as well as humans and intelligent spaces. Visual information plays an important role for developing accurate and useful representation of the static and dynamic states of an intelligent environment. Accurate and efficient capture, analysis, and summarization of the dynamic context requires the vision system to work at multiple levels of semantic abstractions in a robust manner. In this paper, we present details of a long-term and ongoing research project, where indoor intelligent spaces endowed with a range of useful functionalities are designed, built, and systematically evaluated. Some of the key functionalities include: intruder detection; multiple person tracking; body pose and posture analysis; person identification; human body modeling and movement analysis; and for integrated systems for intelligent meeting rooms, teleconferencing, or performance spaces. The paper includes an overall system architecture to support design and development of intelligent environments. Details of panoramic (omnidirectional) video camera arrays, calibration, video stream synchronization, and real-time capture/processing are discussed. Modules for multicamera-based multiperson tracking, event detection and event based servoing for selective attention, voxelization, streaming face recognition, are also discussed. The paper includes experimental studies to systematically evaluate performance of individual video analysis modules as well as to evaluate basic feasibility of an integrated system for dynamic context capture and event based servoing, and semantic information summarization.

Shadow detection algorithms for traffic flow analysis: a comparative study

Shadow detection is critical for robust and reliable vision-based systems for traffic flow analysis. In this paper we discuss various shadow detection approaches and compare two critically. The goal of these algorithms is to prevent moving shadows being misclassified as moving objects (or parts of them), thus avoiding the merging of two or more objects into one and improving the accuracy of object localization. The environment considered is an outdoor highway scene with multiple lanes observed by a single fixed camera. The important features of shadow detection algorithms and the parameter set-up are analyzed and discussed. A critical evaluation of the results both in terms of accuracy and in terms of computational complexity are outlined. Finally, possible integration of the two approaches into a robust shadow detector is presented as future direction of our research.

Articulated body posture estimation from multi-camera voxel data

We present a framework for articulated body model acquisition and tracking from voxel data. A 3D voxel reconstruction of the persons body is computed from silhouettes extracted from four cameras. The model acquisition process is fully automated. In the first frame, body parts are located sequentially. The head is located first, since its shape and size are unique and stable. Other parts are found by sequential template growing and fitting. This initial estimate of body part locations, sizes and orientations is then used as a measurement for the extended Kalman filter which ensures a valid articulated body model. The same filter, with a slightly modified state and state transition matrix, is then used for tracking. The performance of the system has been evaluated on several video sequences with promising results.

Analysis and detection of shadows in video streams: a comparative evaluation

Robustness to changes in illumination conditions as well as viewing perspectives is an important requirement for many computer vision applications. One of the key factors in enhancing the robustness of dynamic scene analysis is that of accurate and reliable means for shadow detection. Shadow detection is critical for correct object detection in image sequences. Many algorithms have been proposed in the literature that deal with shadows. However, a comparative evaluation of the existing approaches is still lacking. In this paper, the full range of problems underlying the shadow detection is identified and discussed. We classify the proposed solutions to this problem using a taxonomy of four main classes, deterministic model and non-model based, and statistical parametric and nonparametric. Novel quantitative (detection and discrimination accuracy) and qualitative metrics (scene and object independence, flexibility to shadow situations and robustness to noise) are proposed to evaluate these classes of algorithms on a benchmark suite of indoor and outdoor video sequences.

Quantifying effects of ligands on androgen receptor nuclear translocation, intranuclear dynamics, and solubility.

Using manual and automated high throughput microscopy (HTM), ligand‐dependent trafficking of green fluorescent protein‐androgen receptor (GFP‐AR) was analyzed in fixed and living cells to determine its spatial distribution, solubility, mobility, and co‐activator interactions. Within minutes, addition of the agonist R1881 resulted translocation of GFP‐AR from the cytoplasm to the nucleus, where it displayed a hyperspeckled pattern and extraction resistance in low expressing cells. AR antagonists (Casodex, hydroxyflutamide) also caused nuclear translocation, however, the antagonist‐bound GFP‐AR had a more diffuse nuclear distribution, distinct from the agonist‐bound GFP‐AR, and was completely soluble; overexpressed GFP‐AR in treated cells was extraction resistant, independent of ligand type. To more dramatically show the different effects of ligand on AR distribution, we utilized an AR with a mutation in the DNA binding domain (ARC619Y) that forms distinct foci upon exposure to agonists but retains a diffuse nuclear distribution in the presence of antagonists. Live‐cell imaging of this mutant demonstrated that cytoplasmic foci formation occurs immediately upon agonist but not antagonist addition. Fluorescence recovery after photobleaching (FRAP) revealed that agonist‐bound GFP‐AR exhibited reduced mobility relative to unliganded or antagonist‐bound GFP‐AR. Importantly, agonist‐bound GFP‐AR mobility was strongly affected by protein expression levels in transiently transfected cells, and displayed reduced mobility even in slightly overexpressing cells. Cyan fluorescent protein‐AR (CFP‐AR) and yellow fluorescent protein‐CREB binding protein (YFP‐CBP) in the presence of agonists and antagonists were used to demonstrate that CFP‐AR specifically co‐localizes with YFP‐CBP in an agonist dependent manner. Dual FRAP experiments demonstrated that CBP mobility mirrored AR mobility only in the presence of agonist. HTM enabled simultaneous studies of the sub‐cellular distribution of GFP‐AR and ARC619Y in response to a range of concentrations of agonists and antagonists (ranging from 10−12 to 10−5) in thousands of cells. These results further support the notion that ligand specific interactions rapidly affect receptor and co‐factor organization, solubility, and molecular dynamics, and each can be aberrantly affected by mutation and overexpression. J. Cell. Biochem. 98: 770–788, 2006.

Activity monitoring and summarization for an intelligent meeting room

Intelligent meeting rooms should support efficient and effective interactions among their occupants. In this paper, we present our efforts toward building intelligent environments using a multimodal sensor network of static cameras, active (pan/tilt/zoom) cameras and microphone arrays. Active cameras are used to capture details associated with interesting events. The goal is not only to make a system that supports multi-person interactions in the environment in real time, but also to have the system remember the past, enabling reviews of past events in an intuitive and efficient manner. In this paper, we present the system specifications and major components, integration framework, active network control procedures and experimental studies involving multi-person interactions in an intelligent meeting room environment.

Intelligent environments and active camera networks

Intelligent environments provide challenging research problems for natural and efficient interfaces between humans and computers as well as between humans. We present a multimodal sensory intelligent system testbed based on some general requirements for developing intelligent environments. We also present rigorous experimental investigations on the processing and control modules for the active camera networks and the microphone array which are embedded in the intelligent room. An integrated intelligent system is developed utilizing four basic modules for visual and audio processing. The integrated system has the functionality of human tracking, active camera control, face recognition and speaker recognition. This system is demonstrated to be suitable for teleconferencing type of applications.