Rachel Alter-Gartenberg

Information-theoretic assessment of sampled imaging systems

By rigorously extending modern communication theory to the assessment of sampled imaging systems, we develop the formulations that are required to optimize the performance of these systems within the critical constraints of image gathering, data transmission, and image display. The goal of this optimization is to produce images with the best possible visual quality for the wide range of statistical properties of the radiance field of natural scenes that one normally encounters. Extensive computational results are presented to assess the performance of sampled imaging systems in terms of information rate, theoretical minimum data rate, and fidelity. Comparisons of this assessment with perceptual and measurable performance demonstrate that (1) the information rate that a sampled imaging system conveys from the captured radiance field to the observer is closely correlated with the fidelity, sharpness and clarity with which the observed images can be restored and (2) the associated theoretical minimum data rate is closely correlated with the lowest data rate with which the acquired signal can be encoded for efficient transmission.

Image Gathering and Digital Restoration

This paper seeks to unite two disciplines: the electro-optical design of image gathering and display devices and the digital processing for image restoration. So far, these two disciplines have remained independent, following distinctly separate traditions. However, the best possible performance can be attained only when the digital processing algorithm accounts for the critical limiting factors of image gathering and display and the image-gathering device is designed to enhance the performance of the digital-processing algorithm. The following salient advantages accrue: 1. Spatial detail as fine as the sampling interval of the image-gathering device ordinarily can be restored sharply and clearly. 2. Even finer spatial detail than the sampling interval can be restored by combining a multiresponse image-gathering sequence with a restoration filter that properly reassembles the within-passband and aliased signal components. 3. The visual quality produced by traditional image gathering (e.g. television camera) and reconstruction (e.g. cubic convolution) can be improved with a small-kernel restoration operator without an increase in digital processing. 4. The enhancement of radiance-field transitions can be improved for dynamicrange compression (to suppress shadow obscurations) and for edge detection (for computer vision).

Image gathering digital restoration for fidelity and visual quality

Abstract The fidelity and resolution of the traditional Wiener restorations given in the prevalent digital processing literature can be significantly improved when the transformations between the continuous and discrete representations in image gathering and display are accounted for ( J. Opt. Soc. Amer. A 5, 1988, 300–314; 6, 1989, 987–1005). However, the visual quality of these improved restorations also is more sensitive to the defects caused by aliasing artifacts, colored noise, and ringing near sharp edges. In this paper, we characterize these visual defects and present methods for suppressing them. We demonstrate how the visual quality of fidelity-maximized images can be improved when (1) the image-gathering system is specifically designed to enhance the performance of the image-restoration algorithm, and (2) the Wiener filter is combined with interactive Gaussian smoothing, synthetic high edge enhancement, and nonlinear tone-scale transformation. The nonlinear transformation is used primarily to enhance the spatial details that are often obscurred when the normally wide dynamic range of natural radiance fields is compressed into the relatively narrow dynamic range of film and other displays.

Artificial scenes and simulated imaging

A software simulation environment for controlled image processing research is described. The simulation is based on a comprehensive model of the end-to-end imaging process that accounts for statistical characteristics of the scene, image formation, sampling, noise, and display reconstruction. The simulation uses a stochastic process to generate super-resolution digital scenes with variable spatial structure and detail. The simulation of the imaging process accounts for the important components of digital imaging systems, including the transformation from continuous to discrete during acquisition and from discrete to continuous during display. This model is appropriate for a variety of problems that involve image acquisition and display including system design, image restoration, enhancement, compression, and edge detection. By using a model-based simulation, research can be conducted with greater precision, flexibility, and portability than is possible using physical systems and experiments can be replicated on any general-purpose computer.

Visual Communication: Information and Fidelity

This assessment of visual communication deals with image gathering, coding, and restoration as a whole rather than as separate and independent tasks. The approach focuses on two mathematical criteria, information and fidelity, and on their relationships to the entropy of the encoded data and to the visual quality of the restored image. Past applications of these criteria to the assessment of image coding and restoration have been limited to the link that connects the output of the image-gathering device to the input of the image-display device. By contrast, the approach presented in this paper explicitly includes the critical limiting factors that constrain image gathering and display. This extension leads to an end-to-end assessment theory of visual communication that combines optical design with digital processing.

Optimal small kernels for edge detection

An algorithm is developed for defining small kernels that are conditioned on the important components of the imaging process: the nature of the scene, the point-spread function of the image-gathering device, sampling effects, noise, and post-filter interpolation. Subject to constraints on the spatial support of the kernel, the algorithm generates the kernal values that minimize the expected mean-square error of the estimate of the scene characteristic. This development is consistent with the derivation of the spatially unconstrained Wiener characteristic filter, but leads to a small, spatially constrained convolution kernel. Simulation experiments demonstrate that the algorithm is more flexible than traditional small-kernel techniques and yields more accurate estimates.<<ETX>>

Optimal visual communication channels

This paper evaluates optimal quantization and data compression in the context of the end-to-end model for 2-D sampled imaging systems. Results show that the minimum number of bits (data density) required for lossless information transmission depends on the design of the image-gathering device. The information in the acquired signal, not its energy, dictates the trade-off between data transmission and visual quality. The entropy of the encoded signal does not indicate neither the amount of information conveyed by the process nor the preferable design tradeoffs for sampled imaging systems. Optimal end-to-end system design for constraint transmission inevitably involves a trade-off between electronic noise and quantization error. The resulting end-to-end design minimizes the loss of information and maximizes the efficiency of its transfer. >

Nonlinear dynamic range transformation in visual communication channels

The article evaluates nonlinear dynamic range transformation in the context of the end-to-end continuous-input/discrete processing/continuous-display imaging process. Dynamic range transformation is required when we have the following: (i) the wide dynamic range encountered in nature is compressed into the relatively narrow dynamic range of the display, particularly for spatially varying irradiance (e.g., shadow); (ii) coarse quantization is expanded to the wider dynamic range of the display; and (iii) nonlinear tone scale transformation compensates for the correction in the camera amplifier.

Efficient visual communication channels

This paper compares the efficiency of uniform quantization in the spatial domain with frequency dependent quantization in the spatial frequency domain, in the context of the end-to-end performance of visual-communication channels. Results show that the minimum data density required for informationally lossless transmission depends on the design of the image-gathering device. The information in the acquired signal, not the energy, dictates the trade-off between data transmission and visual quality. Frequency dependent quantization that maintains the information capacity of the channel while reducing the entropy of the encoded signal, improves its information efficiency. Information bit-allocation is preferable for optimized visual communication for restoration, whereas energy bit-allocation can be used only for image reconstruction.

Compact image representation by edge primitives

Abstract Bandpassed images, commonly used for edge detection, also retain information about intensities between the edge boundaries. Using the familiar Laplacian-of-Gaussian as a bandpass filter, we present a method to extract and code the edge-associated information (edge primitives) and recover an image representation with high structural fidelity. We demonstrate that the edge-primitives representation is compact and therefore can be coded with high compression ratios.