Ping Xue

Improved estimation for just-noticeable visual distortion

Perceptual visibility threshold estimation, based upon characteristics of the human visual system (HVS), has wide applications in digital image/video processing. An improved scheme for estimating just-noticeable distortion (JND) is proposed in this paper. It is proved to outperform the DCTune model, with the major contributions of a new formula for luminance adaptation adjustment and the incorporation of block classification for contrast masking. The HVS visibility threshold for digital images exhibits an approximately parabolic curve versus gray levels and this has been formulated to yield a more accurate base threshold. Moreover, edge regions have been differentiated via block classification to effectively avoid over-estimation of JND in the said regions. Experiments with different images and the associated subjective tests show improved performance of the proposed scheme over the DCTune model for luminance adaptation (especially in dark regions) and masking effect in edge regions. Our model has further demonstrated to achieve favorable results in perceptual visual distortion gauge and image compression. The improvement in JND estimation facilitates better visual distortion measurement and visual signal compression.

Visual distortion gauge based on discrimination of noticeable contrast changes

This paper presents a method to discriminate pixel differences according to their impact toward perceived visual quality. Noticeable local contrast changes are formulated firstly since contrast is the basic sensory feature in the human visual system (HVS) perception. The analysis aims at quantifying the actual impact of such changes (further divided into increases and decreases on edges) in different signal contexts. An associated full-reference distortion metric proposed next provides better match with the HVS viewing. Experiments have used two independent visual data sets and the related subjective viewing results, and demonstrated the performance improvement of the proposed metric over the relevant existing ones with various video/images and under diversified test conditions. The proposed metric is particularly effective to visual signal with blurring and luminance fluctuations as the major artifacts, and brings about the fundamental improvement when sharpened image edges are involved.

Just-noticeable difference estimation with pixels in images

Perceptual visibility threshold estimation, based upon characteristics of the human visual system (HVS), is widely used in digital image and video processing. We propose in this paper a scheme for estimating JND (just-noticeable difference) with explicit formulation for image pixels, by summing the effects of the visual thresholds in sub-bands. The factors being considered include spatial contrast sensitivity function (CSF), luminance adaptation, and adaptive inter- and intra-band contrast masking. The proposed scheme demonstrates favorable results in noise shaping and perceptual visual distortion gauge for different images, in comparison with the relevant existing JND estimators.

Down-Sampling Based Video Coding Using Super-Resolution Technique

It has been reported that oversampling a still image before compression does not guarantee a good image quality. Similarly, down-sampling before video compression in low bit rate video coding may alleviate the blocking effect and improve peak signal-to-noise ratio of the decoded frames. When the number of discrete cosine transform coefficients is reduced in such a down-sampling based coding (DBC), the bit budget of each coefficient will increase, thus reduce the quantization error. A DBC video coding scheme is proposed in this paper, where a super-resolution technique is employed to restore the down-sampled frames to their original resolutions. The performance improvement of the proposed DBC scheme is analyzed at low bit rates, and verified by experiments.

Saliency Density Maximization for Efficient Visual Objects Discovery

Detection of salient objects in an image remains a challenging problem despite extensive studies in visual saliency, as the generated saliency map is usually noisy and incomplete. In this paper, we propose a new method to discover the salient object without prior knowledge on its shape and size. By searching the sub-image, i.e., a bounding box of maximum saliency density, the new formulation can automatically crop the salient objects of various sizes in spite of the cluttered background, and is capable to handle different types of saliency maps. A global optimal solution is obtained by the proposed density-based branch-and-bound search. The proposed method can apply to both images and videos. Experimental results on a public dataset of 5000 images show that our unsupervised detection approach is comparable to the state-of-the-art learning-based methods. Promising results are also observed in the salient object detection for videos with a good potential in video retargeting.

Improved Super-Resolution Reconstruction From Video

Super-resolution (SR) reconstruction usually consists of four steps: registration, interpolation, restoration, and postprocessing. The registration precision (RP) and the initial SR image estimation (ISIE) greatly influence the quality of reconstructed images. A scheme to enhance RP and ISIE is proposed in this paper. Before the registration, each video frame is iteratively upsampled, the registration from current SR reconstructed frame and its adjacent upsampled frames are then estimated, and adjacent frames are warped with registrations to form the high-definition (HD) constraint set, while input frames are used to construct the low-definition (LD) constraint set. The SR reconstructed image corresponds to the minimum difference with the HD constraint set, and its warped and downsampled form corresponds to the minimum difference with the LD constraint set. ISIE can thus be improved from the HD constraint set. In this scheme, the outlier registration with the HD pixel precision is obtained by comparing warped HD frames with the reconstructed SR image, and the adverse influence can be eliminated in calculating LD difference to accelerate the convergence rate of the SR reconstruction and improve the quality of reconstructed images. The performance improvement of the proposed scheme over some existing work is shown in experimental results

A fast algorithm for rain detection and removal from videos

Detection and removal of rain is important in outdoor surveillance vision systems, since the appearance of rain strikes degrades the performance of various vision-based applications. The existing algorithms address the issue of detecting rain only in the irradiance light field, thus require dozens of successive frames to compute the temporal correlation of rain. Combining the properties of rain in irradiance light field and motion field, this paper presents a new approach for rain detection and removal using only three successive frames. In this approach, motion data are used to differentiate rain from other moving objects. A smoothing method based on anisotropic diffusion is proposed for rain removal. Experimental results verify the efficacy of our algorithm.

Fast Edge-Preserved Postprocessing for Compressed Images

Images are often coded using block-based discrete cosine transform (DCT), where blocking and ringing artifacts are the most common visual distortion. In this letter, a fast algorithm is proposed to alleviate the said artifacts in the DCT domain. The new concept is to decompose a row or column image vector to a gradually changed signal and a fast variational signal, which correspond to low-frequency (LF) and high-frequency (HF) DCT subbands, respectively. Blocking artifacts between adjacent LF blocks are suppressed by smoothing LF components and discarding invalid HF ones, and ringing artifacts inside HF vectors are reduced by a simplified bilateral filter. With such a process, edges are preserved while blockiness and ringing are alleviated. Analytic and experimental results confirm the robustness and computational efficiency of the proposed method

Saliency density maximization for object detection and localization

Accurate localization of the salient object from an image is a difficult problem when the saliency map is noisy and incomplete. A fast approach to detect salient objects from images is proposed in this paper. To well balance the size of the object and the saliency it contains, the salient object detection is first formulated with the maximum saliency density on the saliency map. To obtain the global optimal solution, a branch-and-bound search algorithm is developed to speed up the detection process. Without any prior knowledge provided, the proposed method can effectively and efficiently detect salient objects from images. Extensive results on different types of saliency maps with a public dataset of five thousand images show the advantages of our approach as compared to some state-of-the-art methods.

Marker-based image segmentation relying on disjoint set union

Abstract Marker-based image segmentation has been widely used in image analysis and understanding. The well-known Meyers marker-based watershed algorithm by immersion is realized using the hierarchical circular queues. A new marker-based segmentation algorithm relying on disjoint set union is proposed in this paper. It consists of three steps, namely: pixel sorting, set union, and pixel resolving. The memory requirement for the proposed algorithm is fixed as 2× N integers ( N is the image size), whereas the memory requirement for Meyers algorithm is image dependent. The advantage of the proposed algorithm lies at its regularity and simplicity in software/firmware/hardware implementation.