Barry Brumitt

Wallflower: principles and practice of background maintenance

Background maintenance is a frequent element of video surveillance systems. We develop Wallflower, a three-component system for background maintenance: the pixel-level component performs Wiener filtering to make probabilistic predictions of the expected background; the region-level component fills in homogeneous regions of foreground objects; and the frame-level component detects sudden, global changes in the image and swaps in better approximations of the background. We compare our system with 8 other background subtraction algorithms. Wallflower is shown to outperform previous algorithms by handling a greater set of the difficult situations that can occur. Finally, we analyze the experimental results and propose normative principles for background maintenance.

EasyLiving: Technologies for Intelligent Environments

The EasyLiving project is concerned with development of an architecture and technologies for intelligent environments which allow the dynamic aggregation of diverse I/O devices into a single coherent user experience. Components of such a system include middleware (to facilitate distributed computing), world modelling (to provide location-based context), perception (to collect information about world state), and service description (to support decomposition of device control, internal logic, and user interface). This paper describes the current research in each of these areas, highlighting some common requirements for any intelligent environment.

Multi-camera multi-person tracking for EasyLiving

While intelligent environments are often cited as a reason for doing work on visual person-tracking, really making an intelligent environment exposes many real-world problems in visual tracking that must be solved to make the technology practical. In the context of our EasyLiving project in intelligent environments, we created a practical person-tracking system that solves most of the real-world problems. It uses two sets of color stereo cameras for tracking multiple people during live demonstrations in a living room. The stereo images are used for locating people, and the color images are used for maintaining their identities. The system runs quickly enough to make the room feel responsive, and it tracks multiple people standing, walking, sitting, occluding, and entering and leaving the space.

Detection and estimation of pointing gestures in dense disparity maps

We describe a real-time system for detecting pointing gestures and estimating the direction of pointing using stereo cameras. Previously, similar systems were implemented using color-based blob trackers, which relied on effective skin color detection; this approach is sensitive to lighting changes and the clothing worn by the user. In contrast, we used a stereo system that produces dense disparity maps in real-time. Disparity maps are considerably less sensitive to lighting changes. Our system subtracts the background, analyzes the foreground pixels to break the body into parts using a robust mixture model, and estimates the direction of pointing. We have tested the system on both coarse and fine pointing by selecting the targets in a room and controlling the cursor on a wall screen, respectively.

Interaction issues in context-aware intelligent environments

Context-aware intelligent environments are computing systems embedded within physical spaces. They are equipped with input and output computing devices for users and sensors to provide contextual information to the system. These environments provide new challenges to interface designers due to a number of differences from typical desktop computing environments, including the lack of a single focal point for the user, a dynamic set of interaction devices, the sensor-rich nature of the environment, the potential of multiple simultaneous users, and the opportunity for diverse interaction modalities. This essay describes these challenges and focuses on issues involving multiple interaction modalities and automatic system behaviors.

Better Living Through Geometry

Abstract: Mark Weiser described ubiquitous computing as “invisible, everywhere computing that does not live on a personal device of any sort, but is in the woodwork everywhere”[1]. The EasyLiving project is concerned with development of an architecture and technologies for ubiquitous computing environments that allow the dynamic aggregation of diverse I/O devices into a single coherent user experience. Though the need for research in distributed computing, perception and interfaces is widely recognised, the importance of an explicit geometric world model for enhancing the user’s experience of a ubiquitous computing system has not been well-articulated. This paper describes three scenarios that benefit from geometric context and introduces the EasyLiving Geometric Model.