Shiqian Wu

Selectively Detail-Enhanced Fusion of Differently Exposed Images With Moving Objects

In this paper, we introduce an exposure fusion scheme for differently exposed images with moving objects. The proposed scheme comprises a ghost removal algorithm in a low dynamic range domain and a selectively detail-enhanced exposure fusion algorithm. The proposed ghost removal algorithm includes a bidirectional normalization-based method for the detection of nonconsistent pixels and a two-round hybrid method for the correction of nonconsistent pixels. Our detail-enhanced exposure fusion algorithm includes a content adaptive bilateral filter, which extracts fine details from all the corrected images simultaneously in gradient domain. The final image is synthesized by selectively adding the extracted fine details to an intermediate image that is generated by fusing all the corrected images via an existing multiscale algorithm. The proposed exposure fusion algorithm allows fine details to be exaggerated while existing exposure fusion algorithms do not provide such an option. The proposed scheme usually outperforms existing exposure fusion schemes when there are moving objects in real scenes. In addition, the proposed ghost removal algorithm is simpler than existing ghost removal algorithms and is suitable for mobile devices with limited computational resource.

Exposure-Robust Alignment of Differently Exposed Images

This letter presents a novel exposure-robust method to align differently exposed images. First, a directional mapping approach is introduced to normalize differently exposed images so as to alleviate the effect of saturation. Then, a non-parametric local binary pattern (LBP) is employed to represent intensity-invariant features of these images. An efficient two-stage alignment is proposed for motion estimation. Experiments on a variety of synthesized and real image sequences demonstrate that the proposed method is less sensitive to the reference image, and robust to 12 exposure values (EV) increments, which is superior to existing methods.

Object tracking based on bit-planes

Abstract. Visual object tracking is one of the most important components in computer vision. The main challenge for robust tracking is to handle illumination change, appearance modification, occlusion, motion blur, and pose variation. But in surveillance videos, factors such as low resolution, high levels of noise, and uneven illumination further increase the difficulty of tracking. To tackle this problem, an object tracking algorithm based on bit-planes is proposed. First, intensity and local binary pattern features represented by bit-planes are used to build two appearance models, respectively. Second, in the neighborhood of the estimated object location, a region that is most similar to the models is detected as the tracked object in the current frame. In the last step, the appearance models are updated with new tracking results in order to deal with environmental and object changes. Experimental results on several challenging video sequences demonstrate the superior performance of our tracker compared with six state-of-the-art tracking algorithms. Additionally, our tracker is more robust to low resolution, uneven illumination, and noisy video sequences.

A mutual local-ternary-pattern based method for aligning differently exposed images

Order feature is the invariant representation of multi-exposed images. However, saturation yield inconsistent order features of multi-exposed images.A novel mutual local ternary pattern (MLTP) is proposed to cope with saturation and large-variation intensities.Image rotation is initially detected by the histogram-based matching.A linear model is derived for fast image registration and coarse-to-fine technique is implemented to cope with large movement. Saturation and large intensity variations occurred in multi-exposed images offer great challenges to align these images. In this paper, a mutual local-ternary-pattern (MLTP) is proposed to represent differently exposed images for image registration. Different from the classical local ternary pattern (LTP) and its variants, the proposed MLTP has two salient properties: (1) The ternary pattern of one image is not only determined by itself, but also relied on its counterpart; (2) The MLTP is grayscale-adaptive. It is analyzed that the proposed MLTP is a good representation to preserve consistency of differently exposed images. Based on the MLTP-coded images, an efficient linear model derived from Taylor expansion is presented to estimate motion parameters. To improve accuracy and efficiency, image rotation is initially detected by the histogram-based matching, and coarse-to-fine technique is implemented to cope with possibly large movement. Extensive experiments carried out on a variety of synthesized and real multi-exposed images demonstrate that the proposed method is robust to 10 exposure values (EV), which is superior to other methods and current commercial HDR tools.

Aligning multi-exposed images: What are the good feature and similarity measure?

High dynamic range (HDR) imaging, a technique to synthesize a sequence of multi-exposed images, has been recently developed to reduce the dynamic range gap between captured images and real scenes. Unlike conventional cases of varying illumination in which each image is best exposed, the multi-exposed images contain severely under/over-exposed regions and have significant variations in intensity, which offer great challenge in image registration. This paper aims to identify what is the invariant representation of multi-exposed images and which similarity measure is good for these images. To this end, we present a comprehensive comparison of existing ordering features and similarity measures. Experimental results show that the mutual information is the best similarity metric, and the median threshold bitmap is the best feature in terms of accuracy and robustness.

Superpixels-based non-local means image denoising

Although the non-local means method achieves high performance in image denoising, its computation is very expensive. In this paper, we propose to reduce the complexity based on superpixel clustering. The idea is to first divide whole image into some superpixels based on pixels similar geometry structure. Then the selection of similar patches is conducted in every superpixel. Experiments show that the proposed method has good denoising performance while the computation complexity is decreased in comparison with other algorithms.

Gaussian Noise Detection and Adaptive Non-local Means Filter

In this paper, a noise adaptive non-local means (NA-NLM) filter is presented to remove additive Gaussian noise from the corrupted images. Firstly, a novel pixel-wise Gaussian noise detection is proposed via eigen features of local Hessian matrix, and a metric is introduced to measure noise strength. Then, image denoising is performed by adaptive NLM filter according to the pixel-wise noise strength, i.e., the NLM filter varies adaptively with the size selections of the search window and similar patches. Experiments carried on Tampere Image Database (TID) demonstrate that the proposed method outperforms the state-of-the-art methods in terms of the peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and subjective visual assessment.

Improving 2D Camera Calibration by LO-RANSAC

The performance of the traditional 2D pattern based camera calibration is affected by sample viewpoints, positions and feature point localization. In this paper, the Locally Optimized RANSAC (Random Sample Consensus) (LO-RANSAC) is employed to remove the unreliable information automatically. To be more particular, a distance between a specific circular point and the underlying image of the absolute conic is adopted, and a local optimization is achieved when the so-far-best model in the RANSAC iterations has been reached. The experiments on artificial and real data demonstrate that the proposed method alleviates the randomness of the RANSAC solution and get more accurate and reliable calibration results than the traditional methods.

New non-negative sparse feature learning approach for content-based image retrieval

One key issue in content-based image retrieval is to extract effective features so as to represent the visual content of an image. In this study, a new non-negative sparse feature learning approach to produce a holistic image representation based on low-level local features is presented. Specifically, a modified spectral clustering method is introduced to learn a non-negative visual dictionary from local features of training images. A non-negative sparse feature encoding method termed non-negative locality-constrained linear coding (NNLLC) is proposed to improve the popular locality-constrained linear coding method so as to obtain more meaningful and interpretable sparse codes for feature representation. Moreover, a new feature pooling strategy named kMaxSum pooling is proposed to alleviate the information loss of the sum pooling or max pooling strategy, which produces a more effective holistic image representation and can be viewed as a generalisation of the sum and max pooling strategies. The retrieval results carried out on two public image databases demonstrate the effectiveness of the proposed approach.

Weighting linear matching for stereo vision

Recently, the local stereo matching algorithms based on the adaptive weighting achieve very accurate disparity maps. Compared to global matching approaches, the local algorithms offer less complexities. However, these methods are still beyond hardware ability for real-time application. In this paper, a novel linear stereo matching algorithm with constant execution time is proposed. To begin with, a weighting linear cost aggregation is introduced based on the Weighting Guided Image Filtering (WGIF) model which can avoid halo artifacts in local filters. Similar to image filtering, `Halos may change edge distribution in disparity map. Moreover, a complete stereo processing pipeline including cost computation, cost aggregation, disparity selection and disparity refinement is constructed. Experimental results shows that the proposed approach is effective and efficient in stereo matching. Compared to other state-of-the-art local algorithms, our approach achieves comparable results while performs better at occlusion edges.