Hasib Ahmed Siddiqui

Training-based demosaicing

Typical digital cameras use a single-chip image sensor covered with a mosaic of red, green, and blue color filters for capturing color information. At each pixel location, only one of the three color values is known. The interpolation of the two missing color values at each pixel in a color filter array image (CFA) is called demosaicing.

Hardware-friendly universal demosaick using non-iterative map reconstruction

Non-Bayer color filter array (CFA) sensors have recently drawn attention due to their superior compression of spectral energy, ability to deliver improved signal-to-noise ratio, or ability to provide high dynamic range (HDR) imaging. Demosaicking methods that perform color interpolation of Bayer CFA data have been widely investigated. However, a bottleneck to the adaption of emerging non-Bayer CFA sensors is the unavailability of efficient color-interpolation algorithms that can demosaick the new patterns. Designing a new demosaick algorithm for every proposed CFA pattern is a challenge. In this paper, we propose a hardware-friendly universal demosaick algorithm based on maximum a-posteriori (MAP) estimation that can be configured to demosaick raw images captured using a variety of CFA sensors. The forward process of mosaicking is modeled as a linear operation. We then use quadratic data-fitting and image prior terms in a MAP framework and pre-compute the inverse matrix for recovering the full RGB image from CFA observations for a given pattern. The pre-computed inverse is later used in real-time application to demosaick the given CFA pattern. The inverse matrix is observed to have a Toeplitz-like structure, allowing for hardware-efficient implementation of the algorithm. We use a set of 24 Kodak color images to evaluate the quality of our demosaick algorithm on three different CFA patterns. The PSNR values of the reconstructed full-channel RGB images from CFA samples are reported in the paper.

Digital ruler: real-time object tracking and dimension measurement using stereo cameras

Stereo metrology involves obtaining spatial estimates of an object’s length or perimeter using the disparity between boundary points. True 3D scene information is required to extract length measurements of an object’s projection onto the 2D image plane. In stereo vision the disparity measurement is highly sensitive to object distance, baseline distance, calibration errors, and relative movement of the left and right demarcation points between successive frames. Therefore a tracking filter is necessary to reduce position error and improve the accuracy of the length measurement to a useful level. A Cartesian coordinate extended Kalman (EKF) filter is designed based on the canonical equations of stereo vision. This filter represents a simple reference design that has not seen much exposure in the literature. A second filter formulated in a modified sensor-disparity (DS) coordinate system is also presented and shown to exhibit lower errors during a simulated experiment.

Touch HDR: photograph enhancement by user controlled wide dynamic range adaptation

High Dynamic Range (HDR) technology enables photographers to capture a greater range of tonal detail. HDR is typically used to bring out detail in a dark foreground object set against a bright background. HDR technologies include multi-frame HDR and single-frame HDR. Multi-frame HDR requires the combination of a sequence of images taken at different exposures. Single-frame HDR requires histogram equalization post-processing of a single image, a technique referred to as local tone mapping (LTM). Images generated using HDR technology can look less natural than their non- HDR counterparts. Sometimes it is only desired to enhance small regions of an original image. For example, it may be desired to enhance the tonal detail of one subject’s face while preserving the original background. The Touch HDR technique described in this paper achieves these goals by enabling selective blending of HDR and non-HDR versions of the same image to create a hybrid image. The HDR version of the image can be generated by either multi-frame or single-frame HDR. Selective blending can be performed as a post-processing step, for example, as a feature of a photo editor application, at any time after the image has been captured. HDR and non-HDR blending is controlled by a weighting surface, which is configured by the user through a sequence of touches on a touchscreen.

Temporal image stacking for noise reduction and dynamic range improvement

The dynamic range of an imager is determined by the ratio of the pixel well capacity to the noise floor. As the scene dynamic range becomes larger than the imager dynamic range, the choices are to saturate some parts of the scene or “bury” others in noise. In this paper we propose an algorithm that produces high dynamic range images by “stacking” sequentially captured frames which reduces the noise and creates additional bits. The frame stacking is done by frame alignment subject to a projective transform and temporal anisotropic diffusion. The noise sources contributing to the noise floor are the sensor heat noise, the quantization noise, and the sensor fixed pattern noise. We demonstrate that by stacking images the quantization and heat noise are reduced and the decrease is limited only by the fixed pattern noise. As the noise is reduced, the resulting cleaner image enables the use of adaptive tone mapping algorithms which render HDR images in an 8-bit container without significant noise increase.

Sparse Fisher's linear discriminant analysis

Fishers linear discriminant analysis (LDA) is traditionally used in statistics and pattern recognition to linearlyproject high-dimensional observations from two or more classes onto a low-dimensional feature space before classification. The computational complexity of the linear feature extraction method increases linearly with dimensionality of the observation samples. For high-dimensional signals, high computational cost can render the method unsuitable for implementation in real time. In this paper, we propose sparse Fishers linear discriminant analysis, which allows one to search for lowdimensional subspaces, spanned by sparse discriminant vectors, in the high-dimensional space of observation samples from two classes. The sparsity constraints on the space of potential discriminant feature vectors are enforced using the sparse matrix transform (SMT) framework, proposed recently for regularized covariance estimation. Classical Fishers LDA is a special case of sparse Fishers LDA when the sparsity constraints on the feature vectors in the estimation algorithm are fully relaxed. The number of non-zero components in a discriminant direction estimated using our proposed discriminant analysis technique is tunable; this feature can be used to control the compromise between computational complexity and accuracy of the eventual classification algorithm. The experimental results discussed in the manuscript demonstrate the effectiveness of the new method for low-complexity data-classification applications.