Zijian Zhu

Selectively Detail-Enhanced Fusion of Differently Exposed Images With Moving Objects

In this paper, we introduce an exposure fusion scheme for differently exposed images with moving objects. The proposed scheme comprises a ghost removal algorithm in a low dynamic range domain and a selectively detail-enhanced exposure fusion algorithm. The proposed ghost removal algorithm includes a bidirectional normalization-based method for the detection of nonconsistent pixels and a two-round hybrid method for the correction of nonconsistent pixels. Our detail-enhanced exposure fusion algorithm includes a content adaptive bilateral filter, which extracts fine details from all the corrected images simultaneously in gradient domain. The final image is synthesized by selectively adding the extracted fine details to an intermediate image that is generated by fusing all the corrected images via an existing multiscale algorithm. The proposed exposure fusion algorithm allows fine details to be exaggerated while existing exposure fusion algorithms do not provide such an option. The proposed scheme usually outperforms existing exposure fusion schemes when there are moving objects in real scenes. In addition, the proposed ghost removal algorithm is simpler than existing ghost removal algorithms and is suitable for mobile devices with limited computational resource.

Movement detection for the synthesis of high dynamic range images

In this paper, we propose an intensity mapping function (IMF) based scheme to detect moving objects in a set of low dynamic range (LDR) images with different known exposure times. The objective is to remove ghosting artifacts from the eventual high dynamic range (HDR) image. Our contributions include a bidirectional similarity detection method, an adaptive threshold for movement detection, and an IMF based method for the synthesis of pixels to fill in the regions of moving objects. Experimental results show that the proposed scheme outperforms existing schemes.

Hybrid Patching for a Sequence of Differently Exposed Images With Moving Objects

It is very challenging to synthesize a high dynamic range (HDR) image from multiple differently exposed low dynamic range images when there are moving objects in the images. This is due to the fact that the moving objects will cause ghosting artifacts to appear in the synthesized HDR image. To prevent such artifacts, a patching algorithm is required to correct motion regions such that all the moving objects are synchronized in the differently exposed images. In this paper, a new optimization problem is formulated to correct the motion regions of the multiple differently exposed images by considering both spatial and temporal consistencies. The resultant scheme is a hybrid patching scheme composed of a correction method which is an intensity mapping function at pixel level, and a hole-filling method that uses block-level template matching. The proposed patching scheme is not only robust to large intensity changes in these input images, but also at regions that are over-or under-exposed. Experimental results show that the proposed method is able to prevent ghosting artifacts from appearing in the final synthesized HDR image.

Exposure-Robust Alignment of Differently Exposed Images

This letter presents a novel exposure-robust method to align differently exposed images. First, a directional mapping approach is introduced to normalize differently exposed images so as to alleviate the effect of saturation. Then, a non-parametric local binary pattern (LBP) is employed to represent intensity-invariant features of these images. An efficient two-stage alignment is proposed for motion estimation. Experiments on a variety of synthesized and real image sequences demonstrate that the proposed method is less sensitive to the reference image, and robust to 12 exposure values (EV) increments, which is superior to existing methods.

Content adaptive bilateral filtering

Bilateral filter is perhaps the most popular and simplest edge-preserving local filter. However, it may produce halos near some edges due to unwanted smoothing of the edges. In this paper, a content adaptive bilateral filter is proposed to overcome this problem. Both spatial similarity and range similarity parameters of the proposed bilateral filter are adaptive to the content of an image to be filtered rather than being fixed as in the existing bilateral filter. The proposed filter preserves edges and smoothes flat areas better than the existing bilateral filter. It is applied to design a local tone mapping algorithm for high dynamic range images as well as a noise reduction algorithm for low dynamic range color images. Experimental results show that the resultant tone mapping algorithm produces images with better visual quality and at the same time halo artifacts are avoided from appearing in the tone mapped image. The resultant noise reduction algorithm can also produce images with higher structural similarity index values and better visual quality. Noise in smooth regions are more reduced and edges are preserved better in the de-noised image.

A bilateral filter in gradient domain

In this paper, a bilateral filter in gradient domain is first proposed. It is then applied to study detail enhancement via multi-light images and noise reduction of differently exposed low dynamic range images. These two applications show that the proposed filter can be applied to extract fine details from a set of images simultaneously and to provide flexibility for noise reduction from selected areas of an image.

Content adaptive guided image filtering

Local filtering based edge-preserving smoothing technique could suffer from halo artifacts when it is applied for image enhancement. In this paper, an adaptive guided image filter is proposed by incorporating edge aware weighting which is derived from normalized local variance of a guidance image into an existing guided filter. It is shown that the proposed filter preserves sharp edges better than the existing guided filter. With the observation, it is applied for tone mapping of high dynamic range images and fusion of differently exposed images. Experimental results show that the resultant tone mapping and exposure fusion algorithms can prevent halo artifacts from appearing in the final images.

Noise reduced high dynamic range tone mapping using information content weights

In this paper, we propose a noise reduced tone mapping method based on information content weights, where the perceptually unimportant pixels are smoothed during the decomposition in two steps. First, a saliency-based information content weight is introduced to give high fidelity to the data term based on the ratio of the local pixel power and the overall noise power in the base layer decomposition. Then, the detail layer is subtracted using the mutual information-based information content weight from the original image luminance and the clean base layer. Experiments show the effectiveness of the proposed method in the improvements of both signal-to-noise ratio and visual quality.

Single image haze removal via a simplified dark channel

Images of outdoor scenes could be degraded by haze, fog, and smoke in the atmosphere. In this paper, we propose a novel single image haze removal algorithm by introducing a minimal color channel and a sky region compensation term. A simplified dark channel is computed via the minimal color channel. The transmission map is first estimated by using the simplified dark channel. To avoid amplifying noise in the sky, a non-negative sky region compensation term is proposed to adjust the transmission map in the sky. The map is then refined via a content adaptive guided image filter and is finally applied to recover the haze image. Experimental results on outdoor images with haze and without haze demonstrate that the proposed algorithm outperforms existing algorithms.

Fast patching of moving regions for high dynamic range imaging

Ghosting artifacts could be generated in a high dynamic range (HDR) image if it is synthesized for a scene with moving objects by using differently exposed low dynamic range (LDR) images. To remove the ghosting artifacts, pixels affected by movement, i.e., moving regions, are detected from all LDR images, and all pixels in the moving regions are not involved in the synthesis of the final HDR image. Besides removing the ghosting artifacts. it is also very important to preserve the dynamic ranges of moving objects. A fast intensity mapping function (IMF) based method is introduced in this poster to patch the moving regions. With the proposed method, all LDR images are involved in the patching of moving regions instead of only one LDR image as in existing methods [Jacobs et al. 2008]. Therefore, the dynamic range of the HDR image is preserved better than existing methods.