Sumana Gupta

Colorization of grayscale images and videos using a semiautomatic approach

Colorization is a computer-aided process of adding color to a grayscale image or video. The task of colorizing a grayscale image involves assigning three dimensional (RGB) pixel values to an image which varies along only one dimension (luminance or intensity). Since different colors may have the same luminance value but vary in hue or saturation, mapping between intensity and color is not unique, and colorization is ambiguous in nature, requiring some amount of human interaction or external information. In this paper we propose a semi-automatic process for colorization where the user indicates how each region should be colored by putting the desired color marker in the interior of the region. The algorithm based on the position and color of the markers, segments the image and colors it. In order to colorize videos, few reference frames are chosen manually from a set of automatically generated key frames and colorized using the above marker approach and their chrominance information is then transferred to the other frames in the video using a color transfer technique making use of motion estimation. The colorization results obtained are visually very good. In addition the amount of manual intervention is reduced since the user only has to apply color markers on few selected reference frames and the proposed algorithm colors the entire video sequence.

Colorization of grayscale images and videos using a semi-automatic approach

Colorization is a computer-aided process of adding color to a grayscale image or video. The task of colorizing a grayscale image involves assigning three dimensional (RGB) pixel values to an image which varies along only one dimension (luminance or intensity). Since different colors may have the same luminance value but vary in hue or saturation, mapping between intensity and color is not unique, and colorization is ambiguous in nature, requiring some amount of human interaction or external information. In this paper we propose a semi-automatic process for colorization where the user indicates how each region should be colored by putting the desired color marker in the interior of the region. The algorithm based on the position and color of the markers, segments the image and colors it. In order to colorize videos, few reference frames are chosen manually from a set of automatically generated key frames and colorized using the above marker approach and their chrominance information is then transferred to the other frames in the video using a color transfer technique making use of motion estimation. The colorization results obtained are visually very good. In addition the amount of manual intervention is reduced since the user only has to apply color markers on few selected reference frames and the proposed algorithm colors the entire video sequence.

Localized image enhancement

Image enhancement is a well established field in image processing. The main objective of image enhancement is to increase the perceptual information contained in an image for better representation using some intermediate steps, like, contrast enhancement, debluring, denoising etc. Among them, contrast enhancement is especially important as human eyes are more sensitive to luminance than the chrominance components of an image. Most of the contrast enhancement algorithms proposed till now are global methods. The major drawback of this global approach is that in practical scenarios, the contrast of an image does not deteriorate uniformly and the outputs of the enhancement techniques reach saturation at proper contrast points. That leads to information loss. In fact, to the best of our knowledge, no non-reference perceptual measure of image quality has yet been proposed to measure localized enhancement. We propose a fast algorithm to increase the contrast of an image locally using singular value decomposition (SVD) approach and attempt to define some parameters which can give clues related to the progress of the enhancement process.

A Fast Nonparametric Noncausal MRF-Based Texture Synthesis Scheme Using a Novel FKDE Algorithm

In this paper, a new algorithm is proposed for fast kernel density estimation (FKDE), based on principal direction divisive partitioning (PDDP) of the data space. A new framework is also developed to apply FKDE algorithms (both proposed and existing), within nonparametric noncausal Markov random field (NNMRF) based texture synthesis algorithm. The goal of the proposed FKDE algorithm is to use the finite support property of kernels for fast estimation of density. It has been shown that hyperplane boundaries for partitioning the data space and principal component vectors of the data space are two requirements for efficient FKDE. The proposed algorithm is compared with the earlier algorithms, with a number of high-dimensional data sets. The error and time complexity analysis, proves the efficiency of the proposed FKDE algorithm compared to the earlier algorithms. Due to the local simulated annealing, direct incorporation of the FKDE algorithms within the NNMRF-based texture synthesis algorithm, is not possible. This work proposes a new methodology to incorporate the effect of local simulated annealing within the FKDE framework. Afterward, the developed texture synthesis algorithms have been tested with a number of different natural textures, taken from a standard database. The comparison in terms of visual similarity and time complexity, between the proposed FKDE based texture synthesis algorithm with the earlier algorithms, show the efficiency.

Background estimation and motion saliency detection using total variation-based video decomposition

As human vision system is highly sensitive to motion present in a scene, motion saliency forms an important feature in a video sequence. Motion information is used for video compression, object segmentation, object tracking and in many other applications. Though its applications are extensive, accurate detection of motion in a given video is complex and computationally expensive for the solutions reported in the literature. Decomposing a video into visually similar and residual videos is a robust way to detect motion salient regions. The existing decomposition techniques require large execution time as the standard form of the problem is NP-hard. We propose a novel algorithm which detects the motion salient regions by decomposing the input video into background and residual videos in much lesser time without sacrificing the accuracy of the decomposition. In addition, the proposed algorithm is completely parallelizable that ensures further reduction in computational time with the use of advanced multicore processors.

High accuracy depth filtering for Kinect using edge guided inpainting

Kinect is an easy and convenient means to calculate the depth of a scene in real time. It is used widely in several applications for its ease of installation and handling. Many of these applications need a high accuracy depth map of the scene for rendering. Unfortunately, the depth map provided by Kinect suffers from various degradations due to occlusion, shadowing, scattering etc. The major two degradations are the edge distortion and shadowing. Edge distortion appears due to the intrinsic properties of Kinect and makes any depth based operation perceptually degraded. The problem of edge distortion removal has not received as much attention as the hole filling problem, though it is considerably important at the post processing stage of a RGB scene. We propose a novel method to remove line distortion in order to construct high accuracy depth map of the scene by exploiting the edge information already present in the RGB image.

Joint indexing and watermarking of video using color information

This paper draws significant attention to the impending need of media copyright protection for the upcoming content based video indexing and retrieval. We propose a unique technique for joint indexing and watermarking of a compressed bitstream. The method employs video segmentation of the compressed bitstream, followed by extraction of key frames. The key frames are the representative frames of the video. The features of the extracted key frames are used both for watermarking as well as an index for retrieval. Watermarking is done on luminance and chrominance samples in the compressed domain bitstream considering the sensitivities of the human visual system(HVS) to luminance and color components. This increases the robustness of the system without increasing the bit rate of the compressed video. The proposed scheme reduces computational complexity to a great extent, and also reduces the storage space.

Visual Saliency Detection Using Spatiotemporal Decomposition

We propose a novel technique for detection of visual saliency in dynamic video based on video decomposition. The decomposition obtains the sparse features in a particular orientation by exploiting the spatiotemporal discontinuities present in a video cube. A weighted sum of the sparse features along three orthogonal directions determines the salient regions in the video cubes. The weights computed using the frame correlation along three directions are based on the characteristic of human visual system that identifies the sparsest feature as the most salient feature in a video. Unlike the existing methods, which detect the salient region as blob, the proposed approach detects the exact boundaries of salient region with minimum false detection. The experimental results confirm that the detected salient regions of a video closely resemble the salient regions detected by actual tracking of human eyes. The algorithm is tested on different types of video contents and compared with the several state-of-the-art methods to establish the effectiveness of the proposed method.

Total variation based fast video decomposition for artifact restoration

Video decomposition into visually similar part and feature part has gained considerable importance due to its applications in different fields of video processing. Some of the diverse problems that can be handled using this decomposition technique are- background estimation, motion saliency detection, single object tracking, multiple object tracking, artifact detection, compression and various others. Though for the technical integrity, video decomposition becomes an obvious tool for video processing, the present approaches provide solutions which are computationally expensive and non-parallelizable. We propose a novel total variation based decomposition method which is an order of magnitude faster than the existing methods and completely parallelizable. We also propose a novel method for reconstructing old color films affected with partial color artifact (PCA) and blotches using the decomposition technique.

Spatio-temporal multi-view synthesis for free viewpoint television

Interest in view synthesis is growing rapidly as it has tremendous applications in free viewpoint television (FTV), 3DTV, games, virtual reality etc. The main problem of view synthesis is that the virtual view contains holes in disoccluded regions. We propose a hole-filling algorithm to fill the disocclusion holes in the virtual view by exploiting the temporal information of the reference views. We also propose an algorithm to avoid the shining of background pixel through a foreground object in the virtual view due to the absence of foreground pixel information. We generate different zoomed views of the scene by applying the concept of view synthesis and observe the variation of holes with different zoom scales. Finally, we propose and demonstrate depth based image segmentation to facilitate parallel computing. Experimental results show that good quality virtual views are generated with high PSNR and with fewer artifacts.