Glenn Healey

Radiometric CCD camera calibration and noise estimation

Changes in measured image irradiance have many physical causes and are the primary cue for several visual processes, such as edge detection and shape from shading. Using physical models for charged-coupled device (CCD) video cameras and material reflectance, we quantify the variation in digitized pixel values that is due to sensor noise and scene variation. This analysis forms the basis of algorithms for camera characterization and calibration and for scene description. Specifically, algorithms are developed for estimating the parameters of camera noise and for calibrating a camera to remove the effects of fixed pattern nonuniformity and spatial variation in dark current. While these techniques have many potential uses, we describe in particular how they can be used to estimate a measure of scene variation. This measure is independent of image irradiance and can be used to identify a surface from a single sensor band over a range of situations. Experimental results confirm that the models presented in this paper are useful for modeling the different sources of variation in real images obtained from video cameras. >

Markov random field models for unsupervised segmentation of textured color images

We present an unsupervised segmentation algorithm which uses Markov random field models for color textures. These models characterize a texture in terms of spatial interaction within each color plane and interaction between different color planes. The models are used by a segmentation algorithm based on agglomerative hierarchical clustering. At the heart of agglomerative clustering is a stepwise optimal merging process that at each iteration maximizes a global performance functional based on the conditional pseudolikelihood of the image. A test for stopping the clustering is applied based on rapid changes in the pseudolikelihood. We provide experimental results that illustrate the advantages of using color texture models and that demonstrate the performance of the segmentation algorithm on color images of natural scenes. Most of the processing during segmentation is local making the algorithm amenable to high performance parallel implementation. >

Models and methods for automated material identification in hyperspectral imagery acquired under unknown illumination and atmospheric conditions

The spectral radiance measured by an airborne imaging spectrometer for a material on the Earths surface depends strongly on the illumination incident of the material and the atmospheric conditions. This dependence has limited the success of material-identification algorithms that rely on hyperspectral image data without associated ground-truth information. In this paper, the authors use a comprehensive physical model to show that the set of observed 0.4-2.5 /spl mu/m spectral-radiance vectors for a material lies in a low-dimensional subspace of the hyperspectral-measurement space. The physical model captures the dependence of the reflected sunlight, reflected skylight, and path-radiance terms on the scene geometry and on the distribution of atmospheric gases and aerosols over a wide range of conditions. Using the subspace model, they develop a local maximum-likelihood algorithm for automated material identification that is invariant to illumination, atmospheric conditions, and the scene geometry. The algorithm requires only the spectral reflectance of the target material as input. The authors show that the low dimensionality of material subspaces allows for the robust discrimination of a large number of materials over a wide range of conditions. They demonstrate the invariant algorithm for the automated identification of material samples in HYDICE imagery acquired under different illumination and atmospheric conditions.

A multiscale representation including opponent color features for texture recognition

We introduce a representation for color texture using unichrome and opponent features computed from Gabor filter outputs. The unichrome features are computed from the spectral bands independently while the opponent features combine information across different spectral bands at different scales. Opponent features are motivated by color opponent mechanisms in human vision. We present a method for efficiently implementing these filters, which is of particular interest for processing the additional information present in color images. Using a data base of 2560 image regions, we show that the multiscale approach using opponent features provides better recognition accuracy than other approaches.

Global color constancy: recognition of objects by use of illumination-invariant properties of color distributions

Color pixel distributions provide a useful cue for object recognition but are dependent on scene illumination. We develop an algorithm that assigns color descriptors to an object that depend on the surface properties of the object and not on the illumination. An object is defined by a set of possibly textured surfaces and gives rise to a color pixel distribution. For a trichromatic system, the algorithm assumes a three-dimensional linear model for surface spectral reflectance. There are no assumptions about the contents of the scene and only weak constraints on the illumination. The global color invariants can be computed in an amount of time that is proportional to the number of pixels that define an object. A set of experiments on complex scenes under various illuminants demonstrates that the global color constancy algorithm performs significantly better than previous recognition algorithms based on color distribution.

Using color for geometry-insensitive segmentation

Physical models indicate that, in general, reflectance is a complicated function of wavelength and geometry. An analysis of general reflectance models, however, shows that approximate reflectance models exist that preserve much of the structure of the more-detailed models. In particular, I show from general models that Shafer’s dichromatic reflection model [ Color Res. Appl.10, 210 ( 1985)] is a reasonable approximation for a large class of inhomogeneous dielectrics. I also show that a unichromatic reflection model is a useful approximation for metals. The approximate color-reflectance model is the basis for two algorithms that use color information. The first algorithm uses normalized color to classify surfaces according to material composition and is insensitive to geometrical variation in the scene. The second algorithm is used to identify metal and dielectric materials from their images. Experimental results are presented.

Segmenting images using normalized color

An algorithm for segmenting images of 3-D scenes is presented. From an input color image, the algorithm determines the number of materials in the scene and labels each pixel according to the corresponding material. This segmentation is useful for many visual tasks including 3-D inspection and 3-D object recognition. The segmentation algorithm is based on a detailed analysis of the physics underlying color image formation and may be applied to images of a wide range of materials and surface textures. An initial edge detection on the intensity image is used to guide the segmentation process and to ensure accurate localization of region boundaries. The algorithm is based on the computation of local image features and can be mapped effectively onto high-performance parallel hardware. Issues related to illumination and sensors are addressed. Experimental results obtained for several images are presented. >

The illumination-invariant recognition of 3D objects using local color invariants

Traditional approaches to three dimensional object recognition exploit the relationship between three dimensional object geometry and two dimensional image geometry. The capability of object recognition systems can be improved by also incorporating information about the color of object surfaces. Using physical models for image formation, the authors derive invariants of local color pixel distributions that are independent of viewpoint and the configuration, intensity, and spectral content of the scene illumination. These invariants capture information about the distribution of spectral reflectance which is intrinsic to a surface and thereby provide substantial discriminatory power for identifying a wide range of surfaces including many textured surfaces. These invariants can be computed efficiently from color image regions without requiring any form of segmentation. The authors have implemented an object recognition system that indexes into a database of models using the invariants and that uses associated geometric information for hypothesis verification and pose estimation. The approach to recognition is based on the computation of local invariants and is therefore relatively insensitive to occlusion. The authors present several examples demonstrating the systems ability to recognize model objects in cluttered scenes independent of object configuration and scene illumination. The discriminatory power of the invariants has been demonstrated by the systems ability to process a large set of regions over complex scenes without generating false hypotheses.

Using Zernike moments for the illumination and geometry invariant classification of multispectral texture

We develop a method for recognizing color texture independent of rotation, scale, and illumination. Color texture is modeled using spatial correlation functions defined within and between sensor bands. Using a linear model for surface spectral reflectance with the same number of parameters as the number of sensor classes, we show that illumination and geometry changes in the scene correspond to a linear transformation of the correlation functions and a linear transformation of their coordinates. A several step algorithm that includes scale estimation and correlation moment computation is used to achieve the invariance. The key to the method is the new result that illumination, rotation, and scale changes in the scene correspond to a specific transformation of correlation function Zernike moment matrices. These matrices can be estimated from a color image. This relationship is used to derive an efficient algorithm for recognition. The algorithm is substantiated using classification results on over 200 images of color textures obtained under various illumination conditions and geometric configurations.

Invariant subpixel material detection in hyperspectral imagery

We present an algorithm for subpixel material detection in hyperspectral data that is invariant to the illumination and atmospheric conditions. The algorithm does not require atmospheric correction. The target material spectral reflectance is the only required prior information. A target material subspace model is constructed from the reflectance using a physical model and a background subspace model is estimated directly from the image. These two subspace models are used to compute maximum-likelihood estimates (MLEs) for the target material component and the background component at each image pixel. These estimates form the basis of a generalized likelihood ratio test for subpixel material detection. We present experimental results, using Hyperspectral Digital Imagery Collection Experiment (HYDICE) imagery, that demonstrate the utility of the algorithm for subpixel material detection under varying illumination and atmospheric conditions.