Donald Tanguay

Understanding performance in coliseum, an immersive videoconferencing system

Coliseum is a multiuser immersive remote teleconferencing system designed to provide collaborative workers the experience of face-to-face meetings from their desktops. Five cameras are attached to each PC display and directed at the participant. From these video streams, view synthesis methods produce arbitrary-perspective renderings of the participant and transmit them to others at interactive rates, currently about 15 frames per second. Combining these renderings in a shared synthetic environment gives the appearance of having all participants interacting in a common space. In this way, Coliseum enables users to share a virtual world, with acquired-image renderings of their appearance replacing the synthetic representations provided by more conventional avatar-populated virtual worlds. The system supports virtual mobility---participants may move around the shared space---and reciprocal gaze, and has been demonstrated in collaborative sessions of up to ten Coliseum workstations, and sessions spanning two continents.Coliseum is a complex software system which pushes commodity computing resources to the limit. We set out to measure the different aspects of resource, network, CPU, memory, and disk usage to uncover the bottlenecks and guide enhancement and control of system performance. Latency is a key component of Quality of Experience for video conferencing. We present how each aspect of the system---cameras, image processing, networking, and display---contributes to total latency. Performance measurement is as complex as the system to which it is applied. We describe several techniques to estimate performance through direct light-weight instrumentation as well as use of realistic end-to-end measures that mimic actual user experience. We describe the various techniques and how they can be used to improve system performance for Coliseum and other network applications. This article summarizes the Coliseum technology and reports on issues related to its performance---its measurement, enhancement, and control.

Computation and performance issues In coliseum: an immersive videoconferencing system

Coliseum is a multiuser immersive remote teleconferencing system designed to provide collaborative workers the experience of face-to-face meetings from their desktops. Five cameras are attached to each PC display and directed at the participant. From these video streams, view synthesis methods produce arbitrary-perspective renderings of the participant and transmit them to others at interactive rates, currently about 15 frames per second. Combining these renderings in a shared synthetic environment gives the appearance of having all participants interacting in a common space. In this way, Coliseum enables users to share a virtual world, with acquired-image renderings of their appearance replacing the synthetic representations provided by more conventional avatar-populated virtual worlds. The system supports virtual mobility--participants may move around the shared space--and reciprocal gaze, and has been demonstrated in collaborative sessions of up to ten Coliseum workstations, and sessions spanning two continents. This paper summarizes the technology, and reports on issues related to its performance.

Practical Methods for Geometric and Photometric Correction of Tiled Projector

We describe a novel, practical method to create largescale, immersive displays by tiling multiple projectors on curved screens. Calibration is performed automatically with imagery from a single uncalibrated camera, without requiring knowledge of the 3D screen shape. Composition of 2D-mesh-based coordinate mappings, from screen-tocamera and from camera-to-projectors, allows image distortions imposed by the screen curvature and camera and projector lenses to be geometrically corrected together in a single non-parametric framework. For screens that are developable surfaces, we show that the screen-to-camera mapping can be determined without some of the complication of prior methods, resulting in a display on which imagery is undistorted, as if physically attached like wallpaper. We also develop a method of photometric calibration that unifies the geometric blending, brightness scaling, and black level offset maps of prior approaches. The functional form of the geometric blending is novel in itself. The resulting method is more tolerant of geometric correction imprecision, so that visual artifacts are significantly reduced at projector edges and overlap regions. Our efficient GPUbased implementation enables a single PC to render multiple high-resolution video streams simultaneously at frame rate to arbitrary screen locations, leaving the CPU largely free to do video decompression and other processing.

A multi-imager camera for variable-definition video (XDTV)

The enabling technologies of increasing PC bus bandwidth, multicore processors, and advanced graphics processors combined with a high-perform-ance multi-image camera system are leading to new ways of considering video. We describe scalable varied-resolution video capture, presenting a novel method of generating multi-resolution dialable-shape panoramas, a line-based calibration method that achieves optimal multi-imager global registration across possibly disjoint views, and a technique for recasting mosaicking homographies for arbitrary planes. Results show synthesis of a 7.5 megapixel (MP) video stream from 22 synchronized uncompressed imagers operating at 30 Hz on a single PC.