Irwin Sobel

A Variational Framework for Retinex

Retinex theory addresses the problem of separating the illumination from the reflectance in a given image and thereby compensating for non-uniform lighting. This is in general an ill-posed problem. In this paper we propose a variational model for the Retinex problem that unifies previous methods. Similar to previous algorithms, it assumes spatial smoothness of the illumination field. In addition, knowledge of the limited dynamic range of the reflectance is used as a constraint in the recovery process. A penalty term is also included, exploiting a-priori knowledge of the nature of the reflectance image. The proposed formulation adopts a Bayesian view point of the estimation problem, which leads to an algebraic regularization term, that contributes to better conditioning of the reconstruction problem.Based on the proposed variational model, we show that the illumination estimation problem can be formulated as a Quadratic Programming optimization problem. An efficient multi-resolution algorithm is proposed. It exploits the spatial correlation in the reflectance and illumination images. Applications of the algorithm to various color images yield promising results.

Space-dependent color gamut mapping: a variational approach

Gamut mapping deals with the need to adjust a color image to fit into the constrained color gamut of a given rendering medium. A typical use for this tool is the reproduction of a color image prior to its printing, such that it exploits best the given printer/medium color gamut, namely the colors the printer can produce on the given medium. Most of the classical gamut mapping methods involve a pixel-by-pixel mapping and ignore the spatial color configuration. Recently proposed spatial-dependent approaches for gamut mapping are either based on heuristic assumptions or involve a high computational cost. In this paper, we present a new variational approach for space-dependent gamut mapping. Our treatment starts with the presentation of a new measure for the problem, closely related to a recent measure proposed for Retinex. We also link our method to recent measures that attempt to couple spectral and spatial perceptual measures. It is shown that the gamut mapping problem leads to a quadratic programming formulation, guaranteed to have a unique solution if the gamut of the target device is convex. An efficient numerical solution is proposed with promising results.

Understanding performance in coliseum, an immersive videoconferencing system

Coliseum is a multiuser immersive remote teleconferencing system designed to provide collaborative workers the experience of face-to-face meetings from their desktops. Five cameras are attached to each PC display and directed at the participant. From these video streams, view synthesis methods produce arbitrary-perspective renderings of the participant and transmit them to others at interactive rates, currently about 15 frames per second. Combining these renderings in a shared synthetic environment gives the appearance of having all participants interacting in a common space. In this way, Coliseum enables users to share a virtual world, with acquired-image renderings of their appearance replacing the synthetic representations provided by more conventional avatar-populated virtual worlds. The system supports virtual mobility---participants may move around the shared space---and reciprocal gaze, and has been demonstrated in collaborative sessions of up to ten Coliseum workstations, and sessions spanning two continents.Coliseum is a complex software system which pushes commodity computing resources to the limit. We set out to measure the different aspects of resource, network, CPU, memory, and disk usage to uncover the bottlenecks and guide enhancement and control of system performance. Latency is a key component of Quality of Experience for video conferencing. We present how each aspect of the system---cameras, image processing, networking, and display---contributes to total latency. Performance measurement is as complex as the system to which it is applied. We describe several techniques to estimate performance through direct light-weight instrumentation as well as use of realistic end-to-end measures that mimic actual user experience. We describe the various techniques and how they can be used to improve system performance for Coliseum and other network applications. This article summarizes the Coliseum technology and reports on issues related to its performance---its measurement, enhancement, and control.

Variational famework for Retinex

Retinex theory addresses the problem of separating the illumination from the reflectance in a given image and thereby compensating for non-uniform lighting. This is in general an ill-posed problem. In this paper we propose a variational model for the Retinex problem that unifies previous methods. Similar to previous algorithms, it assumes spatial smoothness of the illumination field. In addition, knowledge of the limited dynamic range of the reflectance is used as a constraint in the recovery process. A penalty term is also included, exploiting a-priori knowledge of the nature of the reflectance image. The proposed formulation adopts a Bayesian view point of the estimation problem, which leads to an algebraic regularization term, that contributes to better conditioning of the reconstruction problem. Based on the proposed variational model, we show that the illumination estimation problem can be formulated as a Quadratic Programming optimization problem. An efficient multi-resolution algorithm is proposed. It exploits the spatial correlation in the reflectance and illumination images. Applications of the algorithm to various color images yield promising results.

Computation and performance issues In coliseum: an immersive videoconferencing system

Coliseum is a multiuser immersive remote teleconferencing system designed to provide collaborative workers the experience of face-to-face meetings from their desktops. Five cameras are attached to each PC display and directed at the participant. From these video streams, view synthesis methods produce arbitrary-perspective renderings of the participant and transmit them to others at interactive rates, currently about 15 frames per second. Combining these renderings in a shared synthetic environment gives the appearance of having all participants interacting in a common space. In this way, Coliseum enables users to share a virtual world, with acquired-image renderings of their appearance replacing the synthetic representations provided by more conventional avatar-populated virtual worlds. The system supports virtual mobility--participants may move around the shared space--and reciprocal gaze, and has been demonstrated in collaborative sessions of up to ten Coliseum workstations, and sessions spanning two continents. This paper summarizes the technology, and reports on issues related to its performance.

Practical Methods for Geometric and Photometric Correction of Tiled Projector

We describe a novel, practical method to create largescale, immersive displays by tiling multiple projectors on curved screens. Calibration is performed automatically with imagery from a single uncalibrated camera, without requiring knowledge of the 3D screen shape. Composition of 2D-mesh-based coordinate mappings, from screen-tocamera and from camera-to-projectors, allows image distortions imposed by the screen curvature and camera and projector lenses to be geometrically corrected together in a single non-parametric framework. For screens that are developable surfaces, we show that the screen-to-camera mapping can be determined without some of the complication of prior methods, resulting in a display on which imagery is undistorted, as if physically attached like wallpaper. We also develop a method of photometric calibration that unifies the geometric blending, brightness scaling, and black level offset maps of prior approaches. The functional form of the geometric blending is novel in itself. The resulting method is more tolerant of geometric correction imprecision, so that visual artifacts are significantly reduced at projector edges and overlap regions. Our efficient GPUbased implementation enables a single PC to render multiple high-resolution video streams simultaneously at frame rate to arbitrary screen locations, leaving the CPU largely free to do video decompression and other processing.

Recent results in color compositing of three-parameter magnetic resonance scans as a preoperative aid to the management of upper limb sarcomas

We report on improved image quality and some recent clinical results with volume-rendered, color composite, multiparameter, magnetic resonance (MR) datasets and their color histograms. In a previous article we demonstrated the major components of an interative visualization environment for multiparameterT1-,T2-, PD-weighted MR volume datasets. These are compositing of the three parametric volumes, color volume rendering of the resulting composite, interactive segmentation of the tissues based on color cluster statistics derived from display of the three-dimensional (3D) histogram, and head-tracking stereo diplay. We herein review the system design and describe improvements to the quality of the color volume rendering, in the display of the 3D color histogram data, and in the specification of the volume enderer’s opacity function. Finally, we report on some recent clinical results using our system.

Use of 24-bit false-color imagery to enhance visualization of multiparameter MR images

Biomedical image analysis workstations can be linked to 3D data-oriented devices for a new approach to image manipulation in biology and medicine. Stereo monitors allow an intuitive approach to medical diagnosis. The use of 3D head-tracking devices allows a more compelling 3D illusion to be generated. A stylus can be used as an electronic knife for dissecting a 3D data set; furthermore, other 3D sensors are available for tracking operator arm movements. The overall character of this work is firmly application oriented, in order to provide concrete operational tools to the medical user. Such tools range from diagnostic up to therapeutic and robotized use of bioimages.

Optical sectioning microscopy and bioimage-oriented interfaces for 2D/3D and time-variant characterization of biostructures

Bioimage-oriented interfaces are linked to a 3D image formation system, i.e. the optical microscope, for a new approach to image manipulation in biology and medicine. Stereo monitors and 3D head-tracking and sensor devices allow the operator to interact with 3D data sets representing biostructures and to navigate through the object space. Image restoration algorithms have been implemented in order to gather quantitative information and to study the delicate and interesting relationship between structure and function in biological systems.