Wufeng Xue

Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index

It is an important task to faithfully evaluate the perceptual quality of output images in many applications, such as image compression, image restoration, and multimedia streaming. A good image quality assessment (IQA) model should not only deliver high quality prediction accuracy, but also be computationally efficient. The efficiency of IQA metrics is becoming particularly important due to the increasing proliferation of high-volume visual data in high-speed networks. We present a new effective and efficient IQA model, called gradient magnitude similarity deviation (GMSD). The image gradients are sensitive to image distortions, while different local structures in a distorted image suffer different degrees of degradations. This motivates us to explore the use of global variation of gradient based local quality map for overall image quality prediction. We find that the pixel-wise gradient magnitude similarity (GMS) between the reference and distorted images combined with a novel pooling strategy-the standard deviation of the GMS map-can predict accurately perceptual image quality. The resulting GMSD algorithm is much faster than most state-of-the-art IQA methods, and delivers highly competitive prediction accuracy. MATLAB source code of GMSD can be downloaded at http://www4.comp.polyu.edu.hk/~cslzhang/IQA/GMSD/GMSD.htm.

Learning without Human Scores for Blind Image Quality Assessment

General purpose blind image quality assessment (BIQA) has been recently attracting significant attention in the fields of image processing, vision and machine learning. State-of-the-art BIQA methods usually learn to evaluate the image quality by regression from human subjective scores of the training samples. However, these methods need a large number of human scored images for training, and lack an explicit explanation of how the image quality is affected by image local features. An interesting question is then: can we learn for effective BIQA without using human scored images? This paper makes a good effort to answer this question. We partition the distorted images into overlapped patches, and use a percentile pooling strategy to estimate the local quality of each patch. Then a quality-aware clustering (QAC) method is proposed to learn a set of centroids on each quality level. These centroids are then used as a codebook to infer the quality of each patch in a given image, and subsequently a perceptual quality score of the whole image can be obtained. The proposed QAC based BIQA method is simple yet effective. It not only has comparable accuracy to those methods using human scored images in learning, but also has merits such as high linearity to human perception of image quality, real-time implementation and availability of image local quality map.

Blind image quality assessment using joint statistics of gradient magnitude and laplacian features

Blind image quality assessment (BIQA) aims to evaluate the perceptual quality of a distorted image without information regarding its reference image. Existing BIQA models usually predict the image quality by analyzing the image statistics in some transformed domain, e.g., in the discrete cosine transform domain or wavelet domain. Though great progress has been made in recent years, BIQA is still a very challenging task due to the lack of a reference image. Considering that image local contrast features convey important structural information that is closely related to image perceptual quality, we propose a novel BIQA model that utilizes the joint statistics of two types of commonly used local contrast features: 1) the gradient magnitude (GM) map and 2) the Laplacian of Gaussian (LOG) response. We employ an adaptive procedure to jointly normalize the GM and LOG features, and show that the joint statistics of normalized GM and LOG features have desirable properties for the BIQA task. The proposed model is extensively evaluated on three large-scale benchmark databases, and shown to deliver highly competitive performance with state-of-the-art BIQA models, as well as with some well-known full reference image quality assessment models.

Edge Strength Similarity for Image Quality Assessment

The objective image quality assessment aims to model the perceptual fidelity of semantic information between two images. In this letter, we assume that the semantic information of images is fully represented by edge-strength of each pixel and propose an edge-strength-similarity-based image quality metric (ESSIM). Through investigating the characteristics of the edge in images, we define the edge-strength to take both anisotropic regularity and irregularity of the edge into account. The proposed ESSIM is considerably simple, however, it can achieve slightly better performance than the state-of-the-art image quality metrics as evaluated on six subject-rated image databases.

Reduced reference image quality assessment based on Weibull statistics

Theories in fragmentation have proved that the statistics of image gradient magnitude followed a Weibull distribution, with β (scale) and γ (shape) as free parameters, which are demonstrated to be strongly correlated with brain response. In this paper, we chose β extracted from the proposed strongest component map (SCM) in scale space, as the reduced reference (RR) feature, and developed a novel method for reduced reference image quality assessment (RRIQA) named βW-SCM. For each scale, the SCM was constructed by assembling coefficients with maximum amplitude among different orientations into a single map. The Weibull parameters were then estimated from the SCM. The final image quality was computed by summing the geometric mean of the defined absolute and relative deviations of β. Performance evaluation on the well-known LIVE database demonstrated an outstanding advantage of low RR feature data rate with nearly the same prediction accuracy and consistency.

Reduced reference image quality assessment based on statistics of edge

Objective Image Quality Assessment (IQA) model investigation is a hot topic in recent times. This paper proposed a novel and efficient universal Reduced Reference (RR) image quality assessment method based upon the statistics of edge discrimination. Firstly, binary edge maps created from the multi-scale wavelet transform modulus maxima were used as the low level feature to discriminate the difference between the reference and distorted image for IQA purpose. Then the gradient operator was applied on the binary map to produce the so called edge pattern map. The histogram of edge pattern map was used to verify the pattern of the edges of reference and distorted image, respectively. The RR features extracted from the histogram was used to discriminate the difference of edge pattern maps, and then form a new RR IQA model. Comparing to the typical RR model (Zhou Wangs method, 2005), only 12 features (96 bits) are needed instead of 18 features (162 bits) in Zhou Wang et al.s method with better overall performance.

An image quality assessment metric based on Non-shift Edge

In this paper, we propose a novel metric for image quality assessment based on the ratio of Non-shift Edge (rNSE), whose elegance lies in succinctness and effectiveness. In this metric, an image is filtered by the LOG operator, who acts like the classical receptive field, and the edge points are detected as the zero-crossings of the filtered image. Then the binary Non-shift Edge (NSE) map is derived to represent the strong edge structure remained in the distorted image. The perceptual quality is calculated by the ratio of NSE. The performance of rNSE in the scale-threshold plane shows similar frequency and threshold selectivity. Comparing with the existing well-designed metrics, the proposed rNSE performs equivalently in accuracy and consistency.

Perceptual Fidelity Aware Mean Squared Error

How to measure the perceptual quality of natural images is an important problem in low level vision. It is known that the Mean Squared Error (MSE) is not an effective index to describe the perceptual fidelity of images. Numerous perceptual fidelity indices have been developed, while the representatives include the Structural SIMilarity (SSIM) index and its variants. However, most of those perceptual measures are nonlinear, and they cannot be easily dopted as an objective function to minimize in various low level vision tasks. Can MSE be perceptual fidelity aware after some minor adaptation? In this paper we propose a simple framework to enhance the perceptual fidelity awareness of MSE by introducing an l2-norm structural error term to it. Such a Structural MSE (SMSE) can lead to very competitive image quality assessment (IQA) results. More surprisingly, we show that by using certain structure extractors, SMSE can be further turned into a Gaussian smoothed MSE (i.e., the Euclidean distance between the original and distorted images after Gaussian smooth filtering), which is much simpler to calculate but achieves rather better IQA performance than SSIM. The so called Perceptual-fidelity Aware MSE (PAMSE) can have great potentials in applications such as perceptual image coding and perceptual image restoration.

Local binary pattern statistics feature for reduced reference image quality assessment

Measurement of visual quality is of fundamental importance for numerous image and video processing applications. This paper presented a novel and concise reduced reference (RR) image quality assessment method. Statistics of local binary pattern (LBP) is introduced as a similarity measure to form a novel RR image quality assessment (IQA) method for the first time. With this method, first, the test image is decomposed with a multi-scale transform. Second, LBP encoding maps are extracted for each of subband images. Third, the histograms are extracted from the LBP encoding map to form the RR features. In this way, image structure primitive information for RR features extraction can be reduced greatly. Hence, new RR IQA method is formed with only at most 56 RR features. The experimental results on two large scale IQA databases show that the statistic of LBPs is fairly robust and reliable to RR IQA task. The proposed methods show strong correlations with subjective quality evaluations.

IMAGE QUALITY ASSESSMENT BASED ON EDGE

The research on image quality assessment (IQA) has been become a hot topic in most area concerning image processing. Seeking for the efficient IQA model with the neurophysiology support is naturally the goal people put the efforts to pursue. In this paper, we argue that comparing the edges position of reference and distorted image can well measure the image structural distortion and become an efficient IQA metric, while the edge is detected from the primitive structures of image convolving with LOG filters. The proposed metric is called NSER that has been designed following a simple logic based on the cosine distance of the primitive structures and two accessible improvements. Validation is taken by comparison of the well-known state-of-the-art IQA metrics: VIF, MS-SSIM, VSNR over the six IQA databases: LIVE, TID2008, MICT, IVC, A57, and CSIQ. Experiments show that NSER works stably across all the six databases and achieves the good performance.