Qiaohong Li

Blind Image Quality Assessment Based on High Order Statistics Aggregation

Blind image quality assessment (BIQA) research aims to develop a perceptual model to evaluate the quality of distorted images automatically and accurately without access to the non-distorted reference images. The state-of-the-art general purpose BIQA methods can be classified into two categories according to the types of features used. The first includes handcrafted features which rely on the statistical regularities of natural images. These, however, are not suitable for images containing text and artificial graphics. The second includes learning-based features which invariably require large codebook or supervised codebook updating procedures to obtain satisfactory performance. These are time-consuming and not applicable in practice. In this paper, we propose a novel general purpose BIQA method based on high order statistics aggregation (HOSA), requiring only a small codebook. HOSA consists of three steps. First, local normalized image patches are extracted as local features through a regular grid, and a codebook containing 100 codewords is constructed by K-means clustering. In addition to the mean of each cluster, the diagonal covariance and coskewness (i.e., dimension-wise variance and skewness) of clusters are also calculated. Second, each local feature is softly assigned to several nearest clusters and the differences of high order statistics (mean, variance and skewness) between local features and corresponding clusters are softly aggregated to build the global quality aware image representation. Finally, support vector regression is adopted to learn the mapping between perceptual features and subjective opinion scores. The proposed method has been extensively evaluated on ten image databases with both simulated and realistic image distortions, and shows highly competitive performance to the state-of-the-art BIQA methods.

No-Reference Quality Assessment for Multiply-Distorted Images in Gradient Domain

In practice, images available to consumers usually undergo several stages of processing including acquisition, compression, transmission, and presentation, and each stage may introduce certain type of distortion. It is common that images are simultaneously distorted by multiple types of distortions. Most existing objective image quality assessment (IQA) methods have been designed to estimate perceived quality of images corrupted by a single image processing stage. In this letter, we propose a no-reference (NR) IQA method to predict the visual quality of multiply-distorted images based on structural degradation. In the proposed method, a novel structural feature is extracted as the gradient-weighted histogram of local binary pattern (LBP) calculated on the gradient map (GWH-GLBP), which is effective to describe the complex degradation pattern introduced by multiple distortions. Extensive experiments conducted on two public multiply-distorted image databases have demonstrated that the proposed GWH-GLBP metric compares favorably with existing full-reference and NR IQA methods in terms of high accordance with human subjective ratings.

Blind Image Quality Assessment Using Statistical Structural and Luminance Features

Blind image quality assessment (BIQA) aims to develop quantitative measures to automatically and accurately estimate perceptual image quality without any prior information about the reference image. In this paper, we introduce a novel BIQA metric by structural and luminance information, based on the characteristics of human visual perception for distorted image. We extract the perceptual structural features of distorted image by the local binary pattern distribution. Besides, the distribution of normalized luminance magnitudes is extracted to represent the luminance changes in distorted image. After extracting the features for structures and luminance, support vector regression is adopted to model the complex nonlinear relationship from feature space to quality measure. The proposed BIQA model is called no-reference quality assessment using statistical structural and luminance features (NRSL). Extensive experiments conducted on four synthetically distorted image databases and three naturally distorted image databases have demonstrated that the proposed NRSL metric compares favorably with the relevant state-of-the-art BIQA models in terms of high correlation with human subjective ratings. The MATLAB source code and validation results of NRSL are publicly online at http://www.ntu.edu.sg/home/wslin/Publications.htm.

BSD: Blind image quality assessment based on structural degradation

Abstract Research in biological vision and neurology has evidenced that there are separate mechanisms in human visual cortex to process the first- and second-order patterns. Image structures detected by a linear filter are the first-order patterns which describe luminance changes, while patterns that are invisible to linear filters are often referred as the second-order structures. In this paper, we propose a general-purpose blind image quality assessment (BIQA) method by taking account of both the first- and second-order image structures. Specifically, the Prewitt linear filters are used to extract first-order image structures and the local contrast normalization is employed to extract second-order image structures. Perceptual features are extracted from these two image structural maps and used as the input to a support vector regression to model the nonlinear relationship between feature space to human opinion score. Extensive experiments on five image databases manifest the outstanding performance of the proposed method compared to the relevant state-of-the-art BIQA methods.

Non-intrusive quality assessment for enhanced speech signals based on spectro-temporal features

We propose to learn a non-intrusive quality assessment metric for enhanced speech signals. High-dimension spectro-temporal features are extracted by the Gabor filter bank for speech signals. To reduce the high-dimension features, we use PCA (Principal Component Analysis) to process these features. After obtaining the feature vector from audio signals, Support Vector Regression (SVR) is used to learn the metric for quality evaluation of enhanced speech signals. Experimental results on NOIZEUS dataset demonstrate that proposed non-intrusive quality assessment metric by using spectro-temporal features can obtain better performance for enhanced speech signals.

No-reference image quality assessment based on high order derivatives

Research in human visual perception has found that the sense of natural scences cannot be conveyed only through lines and edges. It also needs the knowledge of texture regions within the image, which can be obtained through the analysis of higher derivatives. Inspired by the research from neuroscience that high order derivatives can capture the details of image structure, we propose a novel simple yet effective blind image quality assessment (IQA) metric based on high order derivatives (BHOD). In the proposed metric, we extract multi-scale structural features up to fourth order image derivatives, to obtain the image structural features. Support vector regression (SVR) is used to learn the mapping between feature space and subjective opinion scores. The proposed method is extensively evaluated on three image databases and shows highly competitive performance to state-of-the-art NR-IQA methods.

A Novel Spatial Pooling Strategy for Image Quality Assessment

A variety of existing image quality assessment (IQA) metrics share a similar two-stage framework: at the first stage, a quality map is constructed by comparison between local regions of reference and distorted images; at the second stage, the spatial pooling is adopted to obtain overall quality score. In this work, we propose a novel spatial pooling strategy for image quality assessment through statistical analysis of the quality map. Our in-depth analysis indicates that the overall image quality is sensitive to the quality distribution. Based on the analysis, the quality histogram and statistical descriptors extracted from the quality map are used as input to the support vector regression to obtain the final objective quality score. Experimental results on three large public IQA databases have demonstrated that the proposed spatial pooling strategy can greatly improve the quality prediction performance of the original IQA metrics in terms of correlation with human subjective ratings.

Gradient-weighted structural similarity for image quality assessments

The goal of Image Quality Assessment (IQA) is to design computational models that can automatically predict the perceived image quality consistent with human subjective ratings. In this paper, we propose a full reference IQA metric gradient weighted structural similarity (GW-SSIM) by incorporating the gradient information to the well-known IQA metric SSIM. Experimental results demonstrate that GW-SSIM can greatly improve the quality prediction accuracy and achieve the best performance among the SSIM-based methods by addressing SSIMs shortcomings. Additionally, incorporating the proposed gradient weighting (GW) map into peak-signal-to-noise ratio (PSNR) also makes it quite competitive to state-of-the-art IQA models, and this is meaningful since PSNR is still a widely adopted metric.

Bag-of-words representation for non-intrusive speech quality assessment

Research on non-intrusive speech quality assessment (SQA) aims to develop a computational model simulating the human perception of speech signals accurately and automatically without any prior information about the reference clean speech signals. In this paper, we propose to learn a non-intrusive SQA metric based on bag-of-words (BoW) representation of speech signals. In particular, the proposed method treats the whole speech utterance as a text document and extracts perceptual linear prediction (PLP) features of local segments as words. The speech utterance is then represented as a histogram of codewords, with each entry as the probability of a codeword appeared in the utterance. After the BoW representation of speech signals is obtained, support vector regression (SVR) is used to learn the metric for quality evaluation. Experimental results demonstrate that the proposed non-intrusive SQA metric BoW can obtain better performance than relevant state-of-the-art metrics.

Multiple-Level Feature-Based Measure for Retargeted Image Quality

Objective image retargeting quality assessment aims to use computational models to predict the retargeted image quality consistent with subjective perception. In this paper, we propose a multiple-level feature (MLF)-based quality measure to predict the perceptual quality of retargeted images. We first provide an in-depth analysis on the low-level aspect ratio similarity feature, and then propose a mid-level edge group similarity feature, to better address the shape/structure related distortion. Furthermore, a high-level face block similarity feature is designed to deal with sensitive region deformation. The multiple-level features are complementary as they quantify different aspects of quality degradation in the retargeted image, and the MLF measure learned by regression is used to predict the perceptual quality of retargeted images. Extensive experimental results performed on two public benchmark databases demonstrate that the proposed MLF measure achieves higher quality prediction accuracy than the existing relevant state-of-the-art quality measures.Objective image retargeting quality assessment aims to use computational models to predict the retargeted image quality consistent with subjective perception. In this paper, we propose a multiple-level feature (MLF)-based quality measure to predict the perceptual quality of retargeted images. We first provide an in-depth analysis on the low-level aspect ratio similarity feature, and then propose a mid-level edge group similarity feature, to better address the shape/structure related distortion. Furthermore, a high-level face block similarity feature is designed to deal with sensitive region deformation. The multiple-level features are complementary as they quantify different aspects of quality degradation in the retargeted image, and the MLF measure learned by regression is used to predict the perceptual quality of retargeted images. Extensive experimental results performed on two public benchmark databases demonstrate that the proposed MLF measure achieves higher quality prediction accuracy than the existing relevant state-of-the-art quality measures.