Don Kimber

“I'll get that off the audio”: a case study of salvaging multimedia meeting records

We describe a case study of a complex, ongoing, collaborative work process, where the central activity is a series of meetings reviewing a wide range of subtle technical topics. The problem is the accurate repxting of the results of these meetings, which is the responsibility of a single person, who is not well-versed in all the topics. We provided tools to capture the meeting discussions and tools to “salvage” the cap tured multimedia recordings. Salvaging is a new kind of activity involving replaying, extracting, organizing, and writing. We observed a year of mature salvaging work in the case study. From this we describe the nature of salvage work (the constituent activities, the use of the workspace, the affordances of the audio medium, how practices develop and differentiate, how the content material affects practice). We also demonstrate how this work relates to the larger work processes (the task demands of the setting, the interplay of salvage with capture, the influence on the people being reported on and reported to). Salvaging tools are shown to be valuable for dealing with free-flowing discussions of complex subject matter and for producing high quality documentation.

LiteMinutes: an Internet-based system for multimedia meeting minutes

The Internet provides a highly suitable infrastructure for sharing multimedia meeting records, especially as multimedia technologies become more lightweight and workers more mobile. LiteMinutes is a system that uses both the Web and email for creating, revising, distributing, and accessing multimedia information captured in a meeting. Supported media include text notes taken on wireless laptops, slide images captured from presentations, and video recorded by cameras in the room. At the end of a meeting, text notes are sent by the note taking applet to the server, which formats them in HTML with links from each note item to the captured slide images and video recording. Smart link generation is achieved by capturing contextual metadata such as the on/off state of the media equipment and the room location of the laptop, and inferring whether it makes sense to supply media links to a particular note item. Note takers can easily revise meeting minutes after a meeting by modifying the email message sent to them and mailing it back to the server’s email address. We explore design issues concerning preferences for email and Web access of meeting minutes, as well as the different timeframes for access. We also describe the integration with a comic book style video summary and visualization system with text captions for browsing the video recording of a meeting.

Metabolically Efficient Information Processing

Energy-efficient information transmission may be relevant to biological sensory signal processing as well as to low-power electronic devices. We explore its consequences in two different regimes. In an immediate regime, we argue that the information rate should be maximized subject to a power constraint, and in an exploratory regime, the transmission rate per power cost should be maximized. In the absence of noise, discrete inputs are optimally encoded into Boltzmann distributed output symbols. In the exploratory regime, the partition function of this distribution is numerically equal to 1. The structure of the optimal code is strongly affected by noise in the transmission channel. The Arimoto-Blahut algorithm, generalized for cost constraints, can be used to derive and interpret the distribution of symbols for optimal energy-efficient coding in the presence of noise. We outline the possibilities and problems in extending our results to information coding and transmission in neurobiological systems.

FlyAbout: spatially indexed panoramic video

We describe a system called FlyAbout which uses spatially indexed panoramic video for virtual reality applications. Panoramic video is captured by moving a 360@deg camera along continuous paths. Users can interactively replay the video with the ability to view any interesting object or choose a particular direction. Spatially indexed video gives the ability to travel along paths or roads with a map-like interface. At junctions, or intersection points, users can chose which path to follow as well as which direction to look, allowing interaction not available with conventional video. Combining the spatial index with a spatial databsdde of maps or objects allows users to navigate to specific locations or interactively inspect particular objects.

FlySPEC: a multi-user video camera system with hybrid human and automatic control

FlySPEC is a video camera system designed for real-time remote operation. A hybrid design combines the high resolution of an optomechanical video camera with the wide field of view always available from a panoramic camera. The control system integrates requests from multiple users so that each controls a virtual camera. The control system seamlessly integrates manual and fully automatic control. It supports a range of options from untended automatic to full manual control. The system can also learn control strategies from user requests. Additionally, the panoramic view is always available for an intuitive interface, and objects are never out of view regardless of the zoom factor. We present the system architecture, an information-theoretic approach to combining panoramic and zoomed images to optimally satisfy user requests, and experimental results that show the FlySPEC system significantly assists users in a remote inspection tasks.

DOTS: support for effective video surveillance

DOTS (Dynamic Object Tracking System) is an indoor, real-time, multi-camera surveillance system, deployed in a real office setting. DOTS combines video analysis and user interface components to enable security personnel to effectively monitor views of interest and to perform tasks such as tracking a person. The video analysis component performs feature-level foreground segmentation with reliable results even under complex conditions. It incorporates an efficient greedy-search approach for tracking multiple people through occlusion and combines results from individual cameras into multi-camera trajectories. The user interface draws the users. attention to important events that are indexed for easy reference at a later time. Different views within the user interface provide spatial information for easier navigation. Our system, with over twenty video cameras installed in hallways and other public spaces in our office building, has been in constant use for almost a year.

Context and observation driven latent variable model for human pose estimation

Current approaches to pose estimation and tracking can be classified into two categories: generative and discriminative. While generative approaches can accurately determine human pose from image observations, they are computationally expensive due to search in the high dimensional human pose space. On the other hand, discriminative approaches do not generalize well, but are computationally efficient. We present a hybrid model that combines the strengths of the two in an integrated learning and inference framework. We extend the Gaussian process latent variable model (GPLVM) to include an embedding from observation space (the space of image features) to the latent space. GPLVM is a generative model, but the inclusion of this mapping provides a discriminative component, making the model observation driven. Observation Driven GPLVM (OD-GPLVM) not only provides a faster inference approach, but also more accurate estimates (compared to GPLVM) in cases where dynamics are not sufficient for the initialization of search in the latent space. We also extend OD-GPLVM to learn and estimate poses from parameterized actions/gestures. Parameterized gestures are actions which exhibit large systematic variation in joint angle space for different instances due to difference in contextual variables. For example, the joint angles in a forehand tennis shot are function of the height of the ball (Figure 2). We learn these systematic variations as a function of the contextual variables. We then present an approach to use information from scene/objects to provide context for human pose estimation for such parameterized actions.

Panoramic video capturing and compressed domain virtual camera control

A system for capturing panoramic video and a novel method for corresponding compressed domain virtual camera control is presented. It targets applications such as classroom lectures and video conferencing. The proposed method is based on the FlyCam panoramic video system that is designed to produce high resolution and wide-angle video sequences by stitching the video pictures from multiple stationary cameras. The panoramic video sequence is compressed into an MPEG-2 stream for delivery. The proposed method integrates region of Interest (ROI) detection, tracking, and virtual camera control, and works on compressed domain information only. It first detects the ROI in the P (predictive coded) picture using only the macroblock type information, It then up-samples this detection result to obtain the ROI of the whole video stream. The ROI is tracked using a Kalman filter. The Kalman filter estimation results are used for virtual camera control that simulates human controlled video recording. The system has no physical camera motion and the virtual camera parameters are readily available for video indexing. The proposed system has been implemented for real time processing.

FlyCam: practical panoramic video

This demonstration presents a computationally and materially inexpensive system for panoramic video imaging. Digitally combining images from an array of inexpensive video cameras results in a wide-field panoramic camera, from inexpensive off-the-shelf- hardware. Digital processing both corrects lens distortion and seamlessly merges images into a panoramic video image. Electronically selecting a region of this results in a rapidly steerable “virtual camera.”

Indoor localization using controlled ambient sounds

Audio-based receiver localization in indoor environments has multiple applications including indoor navigation, location tagging, and tracking. Public places like shopping malls and consumer stores often have loudspeakers installed to play music for public entertainment. Similarly, office spaces may have sound conditioning speakers installed to soften other environmental noises. We discuss an approach to leverage this infrastructure to perform audio-based localization of devices requesting localization in such environments, by playing barely audible controlled sounds from multiple speakers at known positions. Our approach can be used to localize devices such as smart-phones, tablets and laptops to sub-meter accuracy. The user does not need to carry any specialized hardware. Unlike acoustic approaches which use high-energy ultrasound waves, the use of barely audible (low energy) signals in our approach poses very different challenges. We discuss these challenges, how we addressed those, and experimental results on two prototypical implementations: a request-play-record localizer, and a continuous tracker. We evaluated our approach in a real world meeting room and report promising initial results with localization accuracy within half a meter 94% of the time. The system has been deployed in multiple zones in our office building and is used on a regular basis.