Andreas Rossholm

No-reference image and video quality assessment: a classification and review of recent approaches

The field of perceptual quality assessment has gone through a wide range of developments and it is still growing. In particular, the area of no-reference (NR) image and video quality assessment has progressed rapidly during the last decade. In this article, we present a classification and review of latest published research work in the area of NR image and video quality assessment. The NR methods of visual quality assessment considered for review are structured into categories and subcategories based on the types of methodologies used for the underlying processing employed for quality estimation. Overall, the classification has been done into three categories, namely, pixel-based methods, bitstream-based methods, and hybrid methods of the aforementioned two categories. We believe that the review presented in this article will be helpful for practitioners as well as for researchers to keep abreast of the recent developments in the area of NR image and video quality assessment. This article can be used for various purposes such as gaining a structured overview of the field and to carry out performance comparisons for the state-of-the-art methods.

A reduced complexity no-reference artificial neural network based video quality predictor

There is a growing need for robust methods for reference free perceptual quality measurements due to the increasing use of video in hand-held multimedia devices. These methods are supposed to consider pertinent artifacts introduced by the compression algorithm selected for source coding. This paper proposes a model that uses readily available encoder parameters as input to an artificial neural network to predict objective quality metrics for compressed video without using any reference and without need for decoding. The results verify its robustness for prediction of objective quality metrics in general and for PEVQ and PSNR in particular. The paper also focuses on reducing the complexity of the neural network.

Subjective quality assessment of H.264/AVC encoded low resolution videos

Advancements in the video processing area have been proliferated by services that require low delay. Such services involve applications being offered at various temporal and spatial resolutions. It necessitates to study the impacts of related video coding conditions upon perceptual quality. But most of studies concerned with quality assessment of videos affected by coding distortions lack in variety of spatio-temporal resolutions. This paper presents a work done on quality assessment of videos encoded by state-of-the-art H.264/AVC standard at different bitrates and frame rates. Overall, 120 test scenarios for video sequences having different spatial and temporal spectral information were studied. The used coded bistreams in this work and the corresponding subjective assessment scores have been made public for the research community to facilitate further studies.

A no-reference machine learning based video quality predictor

The growing need of quick and online estimation of video quality necessitates the study of new frontiers in the area of no-reference visual quality assessment. Bitstream-layer model based video quality predictors use certain visual quality relevant features from the encoded video bitstream to estimate the quality. Contemporary techniques vary in the number and nature of features employed and the use of prediction model. This paper proposes a prediction model with a concise set of bitstream based features and a machine learning based quality predictor. Several full reference quality metrics are predicted using the proposed model with reasonably good levels of accuracy, monotonicity and consistency.

Assessment of the rating performance of ITU-T recommended video quality metrics in the context of video freezes

Video streaming and multimedia applications are getting popular with the growth of networks. In real-time video streaming, video quality can be degraded due to network performance issues. Among other artifacts, freezing and frame dropping are factors that influence user experience. Service providers, operators, and researchers are interested to measure the Quality of Experience objectively. Different algorithms have been proposed and implemented in this regard. Some of them are in the recommendation list of the ITU Telecommunication Standardization Sector (ITU-T). In this paper, we study the effect of the freezing artifact on user experience and compare the mean opinion score (MOS) of these videos with the results of two algorithms, Perceptual Evaluation of Video Quality (PEVQ) and Temporal Quality Metric, both being part of ITU-T Recommendation J.247 Annex B and C, respectively. Another contribution of this paper is the investigation of the impact of different resolutions and frame rates on user experience.

Perceptual quality estimation of H.264/AVC videos using reduced-reference and no-reference models

Abstract. Reduced-reference (RR) and no-reference (NR) models for video quality estimation, using features that account for the impact of coding artifacts, spatio-temporal complexity, and packet losses, are proposed. The purpose of this study is to analyze a number of potentially quality-relevant features in order to select the most suitable set of features for building the desired models. The proposed sets of features have not been used in the literature and some of the features are used for the first time in this study. The features are employed by the least absolute shrinkage and selection operator (LASSO), which selects only the most influential of them toward perceptual quality. For comparison, we apply feature selection in the complete feature sets and ridge regression on the reduced sets. The models are validated using a database of H.264/AVC encoded videos that were subjectively assessed for quality in an ITU-T compliant laboratory. We infer that just two features selected by RR LASSO and two bitstream-based features selected by NR LASSO are able to estimate perceptual quality with high accuracy, higher than that of ridge, which uses more features. The comparisons with competing works and two full-reference metrics also verify the superiority of our models.

QoE rating performance evaluation of ITU-T recommended video quality metrics in the context of video freezes

In real-time video streaming, video quality can be degraded due to network performance issues. Among other artefacts, video freezing and video jumping are factors that influence user experience. Service providers, operators and manufacturers are interested in evaluating the quality of experience (QoE) objectively because subjective assessment of QoE is expensive and, in many user cases, subjective assessment is not possible to perform. Different algorithms have been proposed and implemented in this regard. Some of them are in the recommendation list of the ITU Telecommunication Standardization Sector (ITU-T). In this paper, we study the effect of the freezing artefact on user experience and compare the mean opinion score of these videos with the results of two algorithms, the perceptual evaluation of video quality (PEVQ) and temporal quality metric (TQM). Both metrics are part of the ITU-T Recommendation J.247 Annex B and C. PEVQ is a full-reference video quality metric, whereas TQM is a no-reference quality metric. Another contribution of this paper is the study of the impact of different resolutions and frame rates on user experience and how accurately PEVQ and TQM measure varying frame rates.

A high quality adjustable complexity motion estimation algorithm for video encoders

In the video encoding process, the motion estimation usually consumes a large part of the encoder computations. This paper presents motion estimation techniques, targeted mainly for MPEG-4 video encoding but also applicable for other video codecs e.g. H.264. A high quality adaptive algorithm with adjustable complexity, based on partially blind prediction for motion estimation, is proposed. The computational complexity of motion estimation is reduced with minor loss in the video quality. In the paper, the quality metrics PSNR, BD PSNR and PEVQ are used, and the possible trade off between complexity and visual quality is studied.

A robust method for estimating synchronization and delay of audio and video for communication services

One of the main contributions to the quality of experience in streaming services or in two-way communication of audio and video applications is synchronization. This has been shown in several studies and experiments but methods to measure synchronization are less frequent, especially for situations without internal access to the application and independent of platform and device. In this paper we present a method for measuring synchronization skewness as well as delay for audio and video. The solution incorporates audio and video reference streams, where audio and video frames are marked with frame numbers which are decoded on the receiver side to enable calculation of synchronization and delay. The method has been verified in a two-way communication application in a transparent network with and without inserting known delays, as well as in a network with 5 and 10 % packet loss levels. The method can be used for both streaming and two-way communication services, both with and without access to the internal structures, and enables measurements of applications running on e.g. smartphones, tablets, and laptops under various conditions.

Quality Assessment of an Adaptive Filter for Artifact Reduction in Mobile Video Sequences

In this paper, we examine an adaptive deblocking deringing filter for mobile video sequences in H.263 format. The considered filter has been designed with reference to the constraints of computational complexity and working memory of mobile terminals. The post filter suggested by the International Telecommunications Union (ITU) in Recommendation H.263 App. III is also included as a reference. Given that fidelity metrics such as the peak signal-to-noise ratio (PSNR) do not necessarily correlate well with video quality as experienced by the user, we consider in this paper objective quality metrics that can incorporate knowledge about the users perception into the quality assessment. Guidelines for choosing filter parameters in relation to user-perceived video quality are obtained from the numerical results