Jinshan Pan

Deblurring Text Images via L0-Regularized Intensity and Gradient Prior

We propose a simple yet effective L0-regularized prior based on intensity and gradient for text image deblurring. The proposed image prior is motivated by observing distinct properties of text images. Based on this prior, we develop an efficient optimization method to generate reliable intermediate results for kernel estimation. The proposed method does not require any complex filtering strategies to select salient edges which are critical to the state-of-the-art deblurring algorithms. We discuss the relationship with other deblurring algorithms based on edge selection and provide insight on how to select salient edges in a more principled way. In the final latent image restoration step, we develop a simple method to remove artifacts and render better deblurred images. Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art text image deblurring methods. In addition, we show that the proposed method can be effectively applied to deblur low-illumination images.

Blind Image Deblurring Using Dark Channel Prior

We present a simple and effective blind image deblurring method based on the dark channel prior. Our work is inspired by the interesting observation that the dark channel of blurred images is less sparse. While most image patches in the clean image contain some dark pixels, these pixels are not dark when averaged with neighboring highintensity pixels during the blur process. This change in the sparsity of the dark channel is an inherent property of the blur process, which we both prove mathematically and validate using training data. Therefore, enforcing the sparsity of the dark channel helps blind deblurring on various scenarios, including natural, face, text, and low-illumination images. However, sparsity of the dark channel introduces a non-convex non-linear optimization problem. We introduce a linear approximation of the min operator to compute the dark channel. Our look-up-table-based method converges fast in practice and can be directly extended to non-uniform deblurring. Extensive experiments show that our method achieves state-of-the-art results on deblurring natural images and compares favorably methods that are well-engineered for specific scenarios.

Deblurring Face Images with Exemplars

The human face is one of the most interesting subjects involved in numerous applications. Significant progress has been made towards the image deblurring problem, however, existing generic deblurring methods are not able to achieve satisfying results on blurry face images. The success of the state-of-the-art image deblurring methods stems mainly from implicit or explicit restoration of salient edges for kernel estimation. When there is not much texture in the blurry image (e.g., face images), existing methods are less effective as only few edges can be used for kernel estimation. Moreover, recent methods are usually jeopardized by selecting ambiguous edges, which are imaged from the same edge of the object after blur, for kernel estimation due to local edge selection strategies. In this paper, we address these problems of deblurring face images by exploiting facial structures. We propose a maximum a posteriori (MAP) deblurring algorithm based on an exemplar dataset, without using the coarse-to-fine strategy or ad-hoc edge selections. Extensive evaluations against state-of-the-art methods demonstrate the effectiveness of the proposed algorithm for deblurring face images. We also show the extendability of our method to other specific deblurring tasks.

Kernel estimation from salient structure for robust motion deblurring

Blind image deblurring algorithms have been improving steadily in the past years. Most state-of-the-art algorithms, however, still cannot perform perfectly in challenging cases, especially in large blur setting. In this paper, we focus on how to estimate a good blur kernel from a single blurred image based on the image structure. We found that image details caused by blur could adversely affect the kernel estimation, especially when the blur kernel is large. One effective way to remove these details is to apply image denoising model based on the total variation (TV). First, we developed a novel method for computing image structures based on the TV model, such that the structures undermining the kernel estimation will be removed. Second, we applied a gradient selection method to mitigate the possible adverse effect of salient edges and improve the robustness of kernel estimation. Third, we proposed a novel kernel estimation method, which is capable of removing noise and preserving the continuity in the kernel. Finally, we developed an adaptive weighted spatial prior to preserve sharp edges in latent image restoration. Extensive experiments testify to the effectiveness of our method on various kinds of challenging examples.

Robust Kernel Estimation with Outliers Handling for Image Deblurring

Estimating blur kernels from real world images is a challenging problem as the linear image formation assumption does not hold when significant outliers, such as saturated pixels and non-Gaussian noise, are present. While some existing non-blind deblurring algorithms can deal with outliers to a certain extent, few blind deblurring methods are developed to well estimate the blur kernels from the blurred images with outliers. In this paper, we present an algorithm to address this problem by exploiting reliable edges and removing outliers in the intermediate latent images, thereby estimating blur kernels robustly. We analyze the effects of outliers on kernel estimation and show that most state-of-the-art blind deblurring methods may recover delta kernels when blurred images contain significant outliers. We propose a robust energy function which describes the properties of outliers for the final latent image restoration. Furthermore, we show that the proposed algorithm can be applied to improve existing methods to deblur images with outliers. Extensive experiments on different kinds of challenging blurry images with significant amount of outliers demonstrate the proposed algorithm performs favorably against the state-of-the-art methods.

Soft-Segmentation Guided Object Motion Deblurring

Object motion blur is a challenging problem as the foreground and the background in the scenes undergo different types of image degradation due to movements in various directions and speed. Most object motion deblurring methods address this problem by segmenting blurred images into regions where different kernels are estimated and applied for restoration. Segmentation on blurred images is difficult due to ambiguous pixels between regions, but it plays an important role for object motion deblurring. To address these problems, we propose a novel model for object motion deblurring. The proposed model is developed based on a maximum a posterior formulation in which soft-segmentation is incorporated for object layer estimation. We propose an efficient algorithm to jointly estimate object segmentation and camera motion where each layer can be deblurred well under the guidance of the soft-segmentation. Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art object motion deblurring methods on challenging scenarios.

L0-Regularized Object Representation for Visual Tracking.

In this paper, we propose a robust visual tracking method by L0-regularized prior in a particle filter framework. In contrast to existing methods, the proposed method employs L0 norm to regularize the linear coefficients of incrementally updated linear basis. The sparsity constraint enables the tracker to effectively handle difficult challenges, such as occlusion or image corruption. To achieve realtime processing, we propose a fast and efficient numerical algorithm for solving the proposed L0-regularized model. Although it is an NP-hard problem, the proposed accelerated proximal gradient (APG) approach is guaranteed to converge to a solution quickly. Extensive experimental results on challenging video sequences demonstrate that the proposed method achieves state-of-the-art results both in accuracy and speed.

Motion blur kernel estimation via salient edges and low rank prior

Blind image deblurring, i.e., estimating a blur kernel from a single input blurred image is a severely ill-posed problem. In this paper, we show how to effectively apply low rank prior to blind image deblurring and then propose a new algorithm which combines salient edges and low rank prior. Salient edges provide reliable edge information for kernel estimation, while low rank prior provides data-authentic priors for the latent image. When estimating the kernel, the salient edges are extracted from an intermediate latent image solved by combining the predicted edges and low rank prior, which help preserve more useful edges than previous deconvolution methods do. By solving the blind image deblurring problem in this fashion, high-quality blur kernels can be obtained. Extensive experiments testify to the superiority of the proposed method over state-of-the-art algorithms, both qualitatively and quantitatively.

Saliency detection based on an edge-preserving filter

How to detect visual salient regions is a challenging problem in computer vision. Recently, saliency detection methods that use boundaries or convex hulls under Bayesian framework have attracted lots of attention. Although these methods achieve state-of-the-art results, there still exist some limitations, e.g., the background will get highlighted when the initial convex hulls are not good enough. This paper presents a new algorithm that retains the advantages of such saliency maps while overcoming their shortcomings. First, the initial convex hull is improved by the image matting model which can be efficiently solved by an edge-preserving filter. Second, a more accurate prior map can be obtained by the improved convex hull. Third, the final convex hull is further refined by an edge-preserving filter to compute the observation likelihood. Finally, the Bayesian framework is employed to compute the saliency map. Extensive experiments compared with state-of-the-art saliency detection algorithms demonstrate the effectiveness of our method.

Non-uniform motion deblurring with Kernel grid regularization

Abstract Camera shake during the exposure time often leads to spatially varying blurring effect on images. Existing work usually uses patch based methods that assume the blur in each patch is uniform to solve this problem. However, these kinds of methods do not consider the consistency between different patches and thus leading to inaccurate results with ringing artifacts. In this paper, we propose a Kernel mapping regularized method to solve the non-uniform deblurring problem, where the consistency between image patches is considered to improve blur Kernel estimation. We analyze the theoretical framework of blur Kernels which can be described as a motion path transference, and propose a robust Kernel estimation algorithm based on Earth mover’s distance (Wasserstein metric) to preserve the properties of blur Kernels. In addition, we develop a new Kernel refinement method based on a proposed Ink Dot Diffusion that uses 8 directions of Kernel mapping flow where the erroneous Kernels are identified and corrected. Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art image deblurring methods.