Anustup Choudhury

A Framework for Robust Online Video Contrast Enhancement Using Modularity Optimization

We address the problem of video contrast enhancement. Existing techniques either do not exploit temporal information at all or do not exploit it correctly. This results in inconsistency that causes undesirable flash and flickering artifacts. Our method analyzes video streams and cluster frames that are similar to each other. Our method does not have omniscient information about the entire video sequence. It is an online process with a fixed delay. A sliding window mechanism successfully detects shot boundaries “on-the-fly” in a video. A graph-based technique called “modularity” performs automatic clustering of video frames without a priori information about clusters. For every cluster in the video, we extract key frames belonging to each cluster using eigen analysis and estimate enhancement parameters for only the key frame, then use these parameters to enhance frames belonging to that cluster, thus making our method robust. We evaluate the clustering method on video sequences from the TRECVid 2001 dataset and compare it with existing methods. We show reduction of flash artifacts in enhanced videos. We show statistically significant improvement in perceived video quality and validate that by conducting experiments on human observers. We show application of our clustering process to perform robust video segmentation.

Color Constancy Using Standard Deviation of Color Channels

We address here the problem of color constancy and propose a new method to achieve color constancy based on the statistics of images with color cast. Images with color cast have standard deviation of one color channel significantly different from that of other color channels. This observation is also applicable to local patches of images and ratio of the maximum and minimum standard deviation of color channels of local patches is used as a prior to select a pixel color as illumination color. We provide extensive validation of our method on commonly used datasets having images under varying illumination conditions and show our method to be robust to choice of dataset and at least as good as current state-of-the-art color constancy approaches.

Perceptually motivated automatic sharpness enhancement using hierarchy of non-local means

We address the problem of sharpness enhancement of images. Existing hierarchical techniques that decompose an image into a smooth image and high frequency components based on Gaussian filter and bilateral filter suffer from halo effects, whereas techniques based on weighted least squares extract low contrast features as detail. Other techniques require multiple images and are not tolerant to noise.

Color constancy using denoising methods and cepstral analysis

We address here the problem of color constancy and propose two new methods for achieving color constancy-the first method uses denoising techniques such as a Gaussian filter, Median filter, Bilateral filter and Non-local means filter to smooth the image for illuminant estimation, while the second method acts in the frequency domain by doing a cepstral analysis of the image. We provide extensive validation tests for our illuminant estimation on commonly used datasets having images under different illumination conditions, and the results show that both new methods outperform current state-of-the-art color constancy approaches, at a very low computational cost.

Color contrast enhancement for visually impaired people

We propose an automatic color contrast enhancement algorithm that improves the visual quality of static images for both normally sighted people and for low-vision patients. Existing methods have been shown to work on people either with normal vision or with low vision. Due to the framework of this approach, an enhanced visual experience can be simultaneously provided to normally sighted people and low vision patients.

Hierarchy of nonlocal means for preferred automatic sharpness enhancement and tone mapping

Existing hierarchical techniques that decompose an image into a smooth image and high frequency components based on Gaussian filter and bilateral filter suffer from halo effects, whereas techniques based on weighted least squares extract low contrast features as details. Other techniques require multiple images and are not tolerant to noise. We use a single image to enhance sharpness based on a hierarchical framework using a modified Laplacian pyramid. In order to ensure robustness, we remove noise by using an extra level in the hierarchical framework. We use an edge-preserving nonlocal means filter and modify it to remove potential halo effects and gradient reversals. However, these effects are only reduced but not removed completely after similar modifications are made to the bilateral filter. We compare our results with existing techniques and show better decomposition and enhancement. Based on validation by human observers, we introduce a new measure to quantify sharpness quality, which allows us to automatically set parameters in order to achieve preferred sharpness enhancement. This causes blurry images to be sharpened more and sufficiently sharp images not to be sharpened. Finally, we demonstrate applications in the context of robust high dynamic range tone mapping that is better than state-of-the-art approaches and enhancement of archaeological artifacts.

Image detail enhancement using a dictionary technique

We present a novel approach to detail enhancement using a dictionary-based technique. For each low-resolution input image patch, we seek a sparse representation from an over-complete dictionary and use that to estimate the high-resolution patch. We modify an existing dictionary-based super-resolution method in several ways to achieve enhancement of fine detail without introduction of new artifacts. These modifications include adaptive enhancement of reconstructed detail patches based on edge analysis to avoid halo artifacts and using an adaptive regularization term to enable noise suppression while enhancing detail. We compare with state-of-the-art methods and show better results in terms of enhancement with suppression of noise.

Facial video super resolution using semantic exemplar components

We present a method for video super resolution using exemplar images of semantic components. In previous work, we proposed a novel super resolution framework based on semantic components and applied it to still images of human faces. In this paper, we extend the approach to video sequences and propose several methods to overcome temporal jitter that results from standard single frame processing. To achieve consistent selection of facial components from a database of exemplars, we introduce a weighted histogram constructed over a temporal window. We then use pixel-based alignment between the exemplar and input image to reduce temporal jitter of the selected component. To further improve temporal stability, we include a temporal constraint into a final optimization stage that blends high resolution exemplar image data into the upscaled input image. We compare our results on face video clips to those of several state-of-the-art super resolution methods, demonstrating the efficacy of the proposed approach.

Channeling Mr. Potato head - Face super-resolution using semantic components

We present a novel approach to face super-resolution using information about semantic components. For each facial component, we seek a similar matching aligned component from a dictionary of training face images. During this process, we also consider the appropriate face pose and use information about people wearing glasses. We then blend the gradient and the high-frequency information of the best-matching high-resolution facial components along with glasses to the input image that has been up-scaled by an existing method. In this paper, we compare with several state-of-the-art methods and demonstrate improvements for multiple poses and expressions.

Prediction of HDR quality by combining perceptually transformed display measurements with machine learning

We present an approach to predict overall HDR display quality as a function of key HDR display parameters. We first performed subjective experiments on a high quality HDR display that explored five key HDR display parameters: maximum luminance, minimum luminance, color gamut, bit-depth and local contrast. Subjects rated overall quality for different combinations of these display parameters. We explored two models | a physical model solely based on physically measured display characteristics and a perceptual model that transforms physical parameters using human vision system models. For the perceptual model, we use a family of metrics based on a recently published color volume model (ICT-CP), which consists of the PQ luminance non-linearity (ST2084) and LMS-based opponent color, as well as an estimate of the display point spread function. To predict overall visual quality, we apply linear regression and machine learning techniques such as Multilayer Perceptron, RBF and SVM networks. We use RMSE and Pearson/Spearman correlation coefficients to quantify performance. We found that the perceptual model is better at predicting subjective quality than the physical model and that SVM is better at prediction than linear regression. The significance and contribution of each display parameter was investigated. In addition, we found that combined parameters such as contrast do not improve prediction. Traditional perceptual models were also evaluated and we found that models based on the PQ non-linearity performed better.