Rajeev Ramanath

Demosaicking methods for Bayer color arrays

Digital Still Color Cameras sample the color spectrum using a monolithic array of color filters overlaid on a charge coupled device array such that each pixel samples only one color band. The resulting mosaic of color samples is processed to produce a high resolution color image such that the values of the color bands not sampled at a certain location are estimated from its neighbors. This process is often referred to as demosaicking. This paper introduces and compares a few commonly used demosaicking methods using error metrics like mean squared error in the RGB color space and perceived error in the CIELAB color space.

Color image processing pipeline

Digital still color cameras (DSCs) have gained significant popularity in recent years, with projected sales in the order of 44 million units by the year 2005. Such an explosive demand calls for an understanding of the processing involved and the implementation issues, bearing in mind the otherwise difficult problems these cameras solve. This article presents an overview of the image processing pipeline, first from a signal processing perspective and later from an implementation perspective, along with the tradeoffs involved.

Band selection using independent component analysis for hyperspectral image processing

Although hyperspectral images provide abundant information about objects, their high dimensionality also substantially increases computational burden. Dimensionality reduction offers one approach to Hyperspectral Image (HSI) analysis. Currently, there are two methods to reduce the dimension, band selection and feature extraction. In this paper, we present a band selection method based on Independent Component Analysis (ICA). This method, instead of transforming the original hyperspectral images, evaluates the weight matrix to observe how each band contributes to the ICA unmixing procedure. It compares the average absolute weight coefficients of individual spectral bands and selects bands that contain more information. As a significant benefit, the ICA-based band selection retains most physical features of the spectral profiles given only the observations of hyperspectral images. We compare this method with ICA transformation and Principal Component Analysis (PCA) transformation on classification accuracy. The experimental results show that ICA-based band selection is more effective in dimensionality reduction for HSI analysis.

Binary Tree-based Generic Demosaicking Algorithm for Multispectral Filter Arrays

In this paper, we extend the idea of using mosaicked color filter array (CFA) in color imaging, which has been widely adopted in the digital color camera industry, to the use of multispectral filter array (MSFA) in multispectral imaging. The filter array technique can help reduce the cost, achieve exact registration, and improve the robustness of the imaging system. However, the extension from CFA to MSFA is not straightforward. First, most CFAs only deal with a few bands (3 or 4) within the narrow visual spectral region, while the design of MSFA needs to handle the arrangement of multiple bands (more than 3) across a much wider spectral range. Second, most existing CFA demosaicking algorithms assume the fixed Bayer CFA and are confined to properties only existed in the color domain. Therefore, they cannot be directly applied to multispectral demosaicking. The main challenges faced in multispectral demosaicking is how to design a generic algorithm that can handle the more diversified MSFA patterns, and how to improve performance with a coarser spatial resolution and a less degree of spectral correlation. In this paper, we present a binary tree based generic demosaicking method. Two metrics are used to evaluate the generic algorithm, including the root mean-square error (RMSE) for reconstruction performance and the classification accuracy for target discrimination performance. Experimental results show that the demosaicked images present low RMSE (less than 7) and comparable classification performance as original images. These results support that MSFA technique can be applied to multispectral imaging with unique advantages

Eigenviews for object recognition in multispectral imaging systems

We address the problem of representing multispectral images of objects using eigenviews for recognition purposes. Eigenviews have long been used for object recognition and pose estimation purposes in the grayscale and color image settings. The purpose of this paper is two-fold: firstly to extend the idealogies of eigenviews to multispectral images and secondly to propose the use of dimensionality reduction techniques other than those popularly used. Principal Component Analysis (PCA) and its various kernel-based flavors are popularly used to extract eigenviews. We propose the use of Independent Component Analysis (ICA) and Non-negative Matrix Factorization (NMF) as possible candidates for eigenview extraction. Multispectral images of a collection of 3D objects captured under different viewpoint locations are used to obtain representative views (eigenviews) that encode the information in these images. The idea is illustrated with a collection of eight synthetic objects imaged in both reflection and emission bands. A Nearest Neighbor classifier is used to perform the classification of an arbitrary view of an object. Classifier performance under additive white Gaussian noise is also tested. The results demonstrate that this system holds promise for use in object recognition under the multispectral imaging setting and also for novel dimensionality reduction techniques. The number of eigenviews needed by various techniques to obtain a given classifier accuracy is also calculated as a measure of the performance of the dimensionality reduction technique.

A Generic Binary Tree-Based Progressive Demosaicking Method for Multispectral Filter Array

The technique of multispectral filter array (MSFA) is a multispectral extension of the widely deployed color filter array (CFA), which uses single chip sensors and subsequent interpolation strategies to produce full color images. However, multispectral demosaicking presents unique challenges to traditional CFA demosaicking algorithms which cannot be directly extended to deal with various filter arrays with different numbers of spectral bands or different spatial patterns within each band. In addition, the more spectral bands involved, the more sparse each band samples the image plane, and the less information we can utilize when performing the interpolation. In this paper, we study a generic MSFA demosaicking method, which follows a binary tree structure to determine the order according to which different spectral bands and different pixel locations within each band are progressively interpolated, making the edge information utilized more effectively. The proposed method is demonstrated to outperform three traditional interpolation techniques as well as three advanced CFA demosaicking methods using two performance measures, the root mean square error (RMSE) for reconstruction fidelity and the classification accuracy for target recognition performance.

Generic MSFA mosaicking and demosaicking for multispectral cameras

In this paper, we investigate the potential application of the multispectral filter array (MSFA) techniques in multispectral imaging for reasons like low cost, exact registration, and strong robustness. In both human and many animal visual systems, different types of photoreceptors are organized into mosaic patterns. This behavior has been emulated in the industry to develop the so-called color filter array (CFA) in the manufacture of digital color cameras. In this way, only one color component is measured at each pixel, and the sensed image is a mosaic of different color bands. We extend this idea to multispectral imaging by developing generic mosaicking and demosaicking algorithms. The binary tree-driven MSFA design process guarantees that the pixel distributions of different spectral bands are uniform and highly correlated. These spatial features facilitate the design of the generic demosaicking algorithm based on the same binary tree, which considers three interrelated issues: band selection, pixel selection and interpolation. We evaluate the reconstructed images from two aspects: better reconstruction and better target classification. The experimental results demonstrate that the mosaicking and demosaicking process preserves the image quality effectively, which further supports that the MSFA technique is a feasible solution for multispectral cameras.

Color: Color Models

Color models are used to quantify how to capture colors accurately and how to display them across disparate media. Most importantly, they are used to relate color stimuli to the human experience of color in a meaningful fashion. In this article, we will examine several color models that are commonly used in the field of computational color science. Keywords: color models; perceptual uniformity; device dependent color; device independent color

High-speed video processing and display system

A video processing and display system for performing high speed geometrical image transformations has been designed. It involves looking up the video image by using a pointer memory. The system supports any video format which does not exceed the clock rate that the system supports. It also is capable of changing the brightness and colormap of the image through hardware.

Noise equivalent dimensions in eigenspaces

Eigenspaces are commonly used for dimensionality reduction, either for a compact encoding or for classification of high dimensional data. However in the presence of noise, the eigenspaces that were created using low-noise data, no longer explain the noise-corrupted data. In this paper, we present the notion of noise equivalent dimensions (NED) as a means of increasing the contribution of signal strength to compensate for the contribution of noise. Although the notion of NED is created with reconstruction error in mind, the additional dimensions may very well be used for classification purposes. Experiments with synthetic and real data are presented to demonstrate their use.