Jennifer Schlenzig

Recursive identification of gesture inputs using hidden Markov models

Human-machine interfaces play a role of growing importance as computer technology continues to evolve. Motivated by the desire to provide users with an intuitive gesture input system, we describe the design of a recursive filter applied to the vision-based gesture interpretation problem. The gestures are modeled as a hidden Markov model with the state representing the gesture sequences, and the observations being the current static hand pose. At each time step the recursive filter updates its estimate of what gesture is occurring based on the current extracted pose information. The result is a robust system which provides the user with continual feedback during compound gestures.<<ETX>>

Vision based hand gesture interpretation using recursive estimation

Gesture recognition requires spatio-temporal image sequence analysis. The actual length of the sequence varies with each instantiation of the gesture, and can be quite long in the case of a multiple gesture sequence. To achieve adequate system response we introduce the concept of recursive estimation of the gesture state. This consists of modeling the gestures as a sequence of static hand poses. Using a hidden Markov model where the unobservable state is the spatio-temporal gesture and the hand poses are the observations allows us to determine the current probabilities of each gesture with a finite state estimator. This decomposes the gesture recognition process into two stages: identification of the hand pose within the current image frame and incorporation of the new information into the probability estimates. We illustrate the performance of the estimator by describing the implementation of a telerobotic application.<<ETX>>

Interactive video on WWW: beyond VCR-like interfaces

Abstract The WWW is evolving into a predominantly visual medium. The demand for access to images and video has been increasing rapidly. Interactive Video systems, which provide access to the content in video archives, are starting to emerge on the www. Partly due to the two-dimensional nature of the web, and partly due to the fact that images that comprise the video are two dimensional, most of these systems provide a VCR-like interface (play, fast-forward, reverse, etc., with additions like object selection, motion specification in the image space, and viewpoint selection). The basis of this paper is the realization that the video streams represent projections of a three-dimensional world, and the user is interested in this three-dimensional content and not the actual configuration of pixels in the image space. In this paper, we justify this intuition by enumerating the information-bearing entities that the user is interested in, and the information specification mechanisms that allow the user to query upon these entities. We will describe how such a intuitive system could be implemented using WWW technologies — VRML, HTML, and HTTP — and present our current WWW prototype which is based on extensions to some of these standards. This system is built on top of our multiple perspective interactive video (MPI Video) paradigm which provides a framework for the management of and interactive access to multiple streams of video data capturing different perspectives of related events.

Minimax design of neural net controllers for highly uncertain plants

This paper discusses the use of evolutionary programming (EP) for computer-aided design and testing of neural controllers applied to problems in which the system to be controlled is highly uncertain. Examples include closed-loop control of drug infusion and integrated control of HVAC/lighting/utility systems in large multi-use buildings. The method is described in detail and applied to a modified Cerebellar Model Arithmetic Computer (CMAC) neural network regulator for systems with unknown time delays. The design and testing problem is viewed as a game, in that the controller is chosen with a minimax criterion i.e., minimize the loss associated with its use on the worst possible plant. The technique permits analysis of neural strategies against a set of feasible plants. This yields both the best choice of control parameters and identification of that plant which is most difficult for the best controller to handle.

VRML-based WWW interface to MPI Video

The Virtual Reality Modeling Language, VRML, is fast becoming a de facto standard for representation, interaction with, and exchange of three dimensional scenes. The purpose of this paper is threefold: First, we want to emphasize the role of multiple perspective video streams in automatic creation of dynamic three dimensional scenes and in automatic creation of hypermedia markups on objects in the scene. Second, we want to present VRML as a interaction metaphor for interactive television and video systems. The three-dimensional nature of the VRML representation allows a user the ability to view the real scene from virtually any one of an infinite number of viewpoints. The ability to markup arbitrary three dimensional objects allows the user to interact with the real world and query about parts of it. Thirdly, we want to drive the future VRML specification and development towards onthe-fly updating of three dimensional models, a mode that will be amenable to our approach: modeling the 3D world as a combination of a detailed static world, and the changes computed from per-frame integration of motion information from video data.

Content-centric interactive video on the World Wide Web

Abstract The World Wide Web (WWW) as a mechanism to provide access to “real world” information in the form of live or recorded video and audio data is becoming more common. However, access to this information is limited to simple playback. In this paper, we present an architecture and a WWW implementation of Multiple Perspective Interactive Video, MPI Video , an infrastructure to access these new forms of information in more useful ways. Using an information system to merge the sensory and virtual data into a coherent, accessible, dynamic database, MPI Video provides content-centric interactivity . Multiple users can access this database to retrieve disparate information at the same time. Most interactions occur in a three-dimensional interface (a natural medium for interacting with real world data) which combines relevant real and virtual components. This approach is useful in several applications including teleconferencing, remote monitoring, and interactive entertainment. In this paper, we present the concepts of MPI Video and describe the latest implementation, a Web-based Remote Access, Monitoring, and Presence (RAMP) system.

Use of neural net control strategies in difficult adaptive control problems: Closed loop control of drug infusion

Neural network techniques have been applied to the control of robots [I], They have achieved respectable performance with rather simple structures requiring no underlying model of the underlying system being controlled. They also appear to adapt rather quickly, provided their parameters are correctly set. This paper discusses application of these techniques to the problem of adaptively controlling infusion of drugs during surgery and intensive care. This is a difficult control problem since the system being controlled is non linear, time varying.and inherently unable to be modeled with a-priori information In fact, only very limited information is available to wnstruct a model of the patients response. Three Neural Net Strategies are discussed: ASE[4]. ACE(41 and the CMAC adaptation of Miller.

Minimax design of CMAC encoded neural network controllers using evolutionary programming

The authors describe the use of evolutionary programming for computer-aided design and testing of cerebellar model arithmetic computer (CMAC) encoded neural network regulators. The design and testing problem is viewed as a game in that the controller parameters are to be chosen with a minimax criterion, i.e. to minimize the loss associated with their use on the worst possible plant parameters. The technique permits analysis of neural strategies against a set of plants. This gives both the best choice of control parameters and identification of the plant configuration which is most difficult for the best controller to handle.<<ETX>>