Sourabh Niyogi

Analyzing gait with spatiotemporal surfaces

Human motions generate characteristic spatiotemporal patterns. We have developed a set of techniques for analyzing the patterns generated by people walking across the field of view. After change detection, the XYT pattern can be fit with a smooth spatiotemporal surface. This surface is approximately periodic, reflecting the periodicity of the gait. The surface can be expressed as a combination of a standard parameterized surface-the canonical walk-and a deviation surface that is specific to the individual walk.<<ETX>>

Representation of motion boundaries in retinotopic human visual cortical areas

Edges are important in the interpretation of the retinal image. Although luminance edges have been studied extensively, much less is known about how or where the primate visual system detects boundaries defined by differences in surface properties such as texture, motion or binocular disparity. Here we use functional magnetic resonance imaging (fMRI) to localize human visual cortical activity related to the processing of one such higher-order edge type: motion boundaries. We describe a robust fMRI signal that is selective for motion segmentation. This boundary-specific signal is present, and retinotopically organized, within early visual areas, beginning in the primary visual cortex (area V1). Surprisingly, it is largely absent from the motion-selective area MT/V5 and far extrastriate visual areas. Changes in the surface velocity defining the motion boundaries affect the strength of the fMRI signal. In parallel psychophysical experiments, the perceptual salience of the boundaries shows a similar dependence on surface velocity. These results demonstrate that information for segmenting scenes by relative motion is represented as early as V1.

Example-based head tracking

We want to estimate the pose of human heads. This estimation involves a nonlinear mapping from the input image to an output parametric description. We characterize the mapping through examples from a training set, outputting the pose of the nearest example neighbor of the input. This is vector quantization, with the modification that we store an output parameter code with each quantized input code. For efficient indexing, we use a tree-structured vector quantizer (TSVQ). We make design choices based on the example application of monitoring an automobile drivers face. The reliance on stored data over computation power allows the system to be simple; efficient organization of the data allows it to be fast. We incorporate tracking in position and scale within the same vector quantization framework with virtually no cost in added computation. We show reasonable experimental results for a real-time prototype running on an inexpensive workstation.

Detecting kinetic occlusion

Visual motion boundaries provide a powerful cue for the perceptual organization of scenes. Motion boundaries are present when surfaces in motion occlude one another. Conventional approaches to motion analysis have relied on assumptions of data conservation and smoothness, which has made analysis of motion boundaries difficult. We show that a common source of motion boundary, kinetic occlusion, can be detected using spatiotemporal junction analysis. Junction analysis is accomplished by utilizing distributed representations of motion used in models of human visual motion sensing. By detecting changes in the direction of motion in these representations, spatiotemporal junctions are detected in a manner which differentiates accretion from deletion. We demonstrate successful occlusion detection on spatiotemporal imagery containing occluding surfaces in motion.<<ETX>>

A probabilistic model of theory formation

Concept learning is challenging in part because the meanings of many concepts depend on their relationships to other concepts. Learning these concepts in isolation can be difficult, but we present a model that discovers entire systems of related concepts. These systems can be viewed as simple theories that specify the concepts that exist in a domain, and the laws or principles that relate these concepts. We apply our model to several real-world problems, including learning the structure of kinship systems and learning ontologies. We also compare its predictions to data collected in two behavioral experiments. Experiment 1 shows that our model helps to explain how simple theories are acquired and used for inductive inference. Experiment 2 suggests that our model provides a better account of theory discovery than a more traditional alternative that focuses on features rather than relations.

Spatiotemporal junction analysis for motion boundary detection

Conventional approaches to 2-D motion estimation have relied on assumptions of data conservation and smoothness, which has made analysis of motion boundaries difficult. We propose that one of the cues present at motion boundaries, kinetic occlusion, can be detected using spatiotemporal junction analysis. Junction analysis is accomplished by constructing distributed representations of motion, and spatiotemporally filtering these signals to detect orientation stopping or starting in space-time. Our occlusion detection scheme differentiates accretion from deletion of surface texture at motion boundaries. We demonstrate successful motion boundary extraction on spatiotemporal stimuli containing occluding surfaces in motion.