Renu Rameshan

Joint MAP estimation for blind deconvolution: when does it work?

Blind deconvolution aims at reconstructing an image from its blurred and noisy version, when the blur kernel is not known. It has been acknowledged that the naive maximum aposteriori probability (MAP) algorithm favors a no-blur solution [3]. In [8] the failure of the direct MAP approach is addressed and it is proved that a simultaneous MAP estimation of the image and the point spread function (PSF) fails, providing a trivial solution. In contrast, we show that an appropriate choice of PSF prior during joint MAP estimation does provide a non-trivial solution. We provide the feasible range for the PSF regularization factor which would prevent a trivial solution.

High dynamic range imaging under noisy observations

We propose a radiance domain denoising frame work for the high dynamic range (HDR) imaging problem. The proposed method uses a maximum aposteriori probability (MAP) based reconstruction of the HDR image with total variation (TV) as the prior to avoid unnecessary smoothing of the radiance field. To make the computation with TV prior efficient, we extend the majorize-minimize method of upper bounding the total variation by a quadratic function to our case which has a nonlinear term arising from the camera response function. A theoretical justification for doing radiance domain denoising as opposed to image domain denoising is also provided. Our method yields better results, with the edges well preserved and noise reduced considerably.

Blind Deconvolution Methods: A Review

Researchers have been working on the blind deconvolution problem from as early as 1975 and various methods have been developed over the period. Earlier methods were purely transform domain based, which were modifications of the inverse filter. This was followed by solutions that treat blind deconvolution as an ill-posed problem, thereby using regularization as a tool for solving the problem. Various refinements to this approach that have been proposed are reviewed in this chapter. We also devote a separate section to motion deblur, since there is a renewed interest in this area and a large volume of publications has come up recently in this area.

Sparsity-Based Blind Deconvolution

In the previous chapters our focus was on overcoming the tendency of joint MAP estimator to give trivial solutions by choosing an appropriate PSF regularizer and the regularization factor, and the convergence analysis of the resulting optimization problem. Here we provide an alternate way to avoid the trivial solution in MAP methods by using an image regularizer that has a cost which increases with the amount of blur. We define one such regularizer that is sparsity based and is an improvement over the l1/l2-norm of derivative. We derive a condition to check the suitability of the ratio of l1-norm of derivative to l2-norm of higher order derivatives as image priors. We note that l1/l2-norm of derivative does not exhibit uniform behaviour for all types of images. We classify the images into three types – Type 1, Type 2a, Type 2b, depending on the sparsity of the gradient map and on the strength of the edges. It is noted that only for Type 1 images, which have strong edges, the prior l1/l2-norm of derivative exhibits a monotonic rise in cost function. The regularizer that we have defined exhibits the desired behavior for all types of images including those with sparse and weak edges. We also look into the wavelet domain, which is another sparse domain, and do deconvolution in this domain using the sparsity enforcing l1/l2-norm.

EgoTracker: Pedestrian Tracking with Re-identification in Egocentric Videos

We propose and analyze a novel framework for tracking a pedestrian in egocentric videos, which is needed for analyzing social gatherings recorded with a wearable camera. The constant camera and pedestrian movement makes this a challenging problem. The main challenges are natural head movement of wearer and target loss and reappearance in a later frame, due to frequent changes in field of view. By using the optical flow information specific to egocentric videos and also by modifying the learning process and sampling region of trackers which tracks by learning an SVM online, we show that re-identification is possible. The specific trackers chosen are STRUCK and MEEM.

Object Triggered Egocentric Video Summarization

Egocentric videos are usually of long duration and contains lot of redundancy which makes summarization an essential task for such videos. In this work we are targeting object triggered egocentric video summarization which aims at extracting all the occurrences of an object in a given video, in near real time. We propose a modular pipeline which first aims at limiting the redundant information and then uses a Convolutional Neural Network and LSTM based approach for object detection. Following this we represent the video as a dictionary which captures the semantic information in the video. Matching a query object reduces to doing an And-Or Tree traversal followed by deepmatching algorithm for fine grained matching. The frames containing the object, which would have been missed at the pruning stage are retrieved by running a tracker on the frames selected by the pipeline mentioned. The modular pipeline allows replacing any module with its more efficient version. Performance tests ran on the overall pipeline for egocentric datasets, EDUB dataset and personal recorded videos, give an average recall of 0.76.

Accelerated learning of discriminative spatio-temporal features for action recognition

Recently, paradigm has shifted from hand-designed local feature learning to unsupervised learning in order to extract features from raw data. In action recognition, good results are achieved using deep learning techniques such as stacking and convolution to extend the idea of independent subspace analysis (ISA). Albeit performance is good, it takes significant amount of time on big datasets due to high computational complexity and sequential implementation. We propose two methods for speeding up feature learning using ISA. We also propose input data modification which increases the classification performance. One method of faster feature learning is parallelization - we use the scalable programming model, MapReduce to parametrize ISA algorithm by distributing datasets into equal disjoint sets. The second method for increasing speed is by using spatio-temporal interest point detectors to extract “important” blocks from video. The latter not only enhances the speed but also improves the classification accuracy. We modified input as the gradient of video and achieved a better classification accuracy on all the datasets that were tested. We also created a dataset of water activities and used the ISA network for feature extraction. We achieved speed up by a factor of 4 and 2.4 in first and second method respectively.

Dictionary Based Approach for Facial Expression Recognition from Static Images

We present a simple approach for facial expression recognition from images using the principle of sparse representation using a learned dictionary. Visual appearance based feature descriptors like histogram of oriented gradients (HOG), local binary patterns (LBP) and eigenfaces are used. We use Fisher discrimination dictionary which has discrimination capability in addition to being reconstructive. The classification is based on the fact that each expression class with in the dictionary spans a subspace and these subspaces have non-overlapping directions so that they are widely separated. Each test feature point has a sparse representation in the union of subspaces of dictionary formed by labeled training points. To check recognition performance of the proposed approach, extensive experimentation is done over Jaffee and CK databases. Results show that the proposed approach has better classification accuracy than state-of-the-art techniques.

Conclusions and Future Research Directions

Our focus in this monograph has been on deriving the conditions under which alternate minimization for blind deconvolution yields non-trivial results and on analyzing the convergence of alternating minimization scheme for blind deconvolution. Our findings are summarized in this chapter. We also provide directions for future research in finding appropriate regularizers and also on convergence analysis.

MAP Estimation: When Does It Work?

In this chapter we analyze the performance of the maximum a posteriori probability (MAP) method of estimating the image and the point spread function (PSF), which we term as joint MAP estimation for blind deconvolution since both the unknowns are estimated simultaneously. Many authors have reported the failure of direct application of the MAP estimator in blind deconvolution, the details of which we explain in this chapter. We show that joint MAP estimation fails only when the PSF regularizer is not chosen properly. We also demonstrate that the MAP estimation does produce good results with an appropriate choice of the PSF prior. The emphasis on the word appropriate is very important. We provide a theoretical justification to show when joint MAP estimation works and when it does not. The arguments are substantiated through experimental validation. We also show how the regularization factor is to be selected so that the PSF regularization is effective. Our analysis provides a feasible range for the regularization factor without using cross validation techniques. We give an exact lower bound and an approximate upper bound for the PSF regularization factor.