Andrew Perkis

Perceptual-based quality assessment for audio-visual services: A survey

Accurate measurement of the perceived quality of audio-visual services at the end-user is becoming a crucial issue in digital applications due to the growing demand for compression and transmission of audio-visual services over communication networks. Content providers strive to offer the best quality of experience for customers linked to their different quality of service (QoS) solutions. Therefore, developing accurate, perceptual-based quality metrics is a key requirement in multimedia services. In this paper, we survey state-of-the-art signal-driven perceptual audio and video quality assessment methods independently, and investigate relevant issues in developing joint audio-visual quality metrics. Experiments with respect to subjective quality results have been conducted for analyzing and comparing the performance of the quality metrics. We consider emerging trends in audio-visual quality assessment, and propose feasible solutions for future work in perceptual-based audio-visual quality metrics.

No-reference JPEG-image quality assessment using GAP-RBF

In this paper, we present a novel no-reference (NR) method to assess the quality of JPEG-coded images using a sequential learning algorithm for growing and pruning radial basis function (GAP-RBF) network. The features for predicting the perceived image quality are extracted by considering key human visual sensitivity factors such as edge amplitude, edge length, background activity and background luminance. Image quality estimation involves computation of functional relationship between HVS features and subjective test scores. Here, the functional relationship is approximated using GAP-RBF network. The advantage of using sequential learning algorithm is its capability to learn new samples without affecting the past learning. Further, the sequential learning algorithm requires minimal memory and computational effort. Experimental results prove that the prediction of the trained GAP-RBF network does emulate the mean opinion score (MOS). The subjective test results of the proposed metric are compared with JPEG no-reference image quality index as well as full-reference structural similarity image quality index and it is observed to outperform both.

Perceptual quality assessment based on visual attention analysis

Most existing quality metrics do not take the human attention analysis into account. Attention to particular objects or regions is an important attribute of human vision and perception system in measuring perceived image and video qualities. This paper presents an approach for extracting visual attention regions based on a combination of a bottom-up saliency model and semantic image analysis. The use of PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural SIMilarity) in extracted attention regions is analyzed for image/video quality assessment, and a novel quality metric is proposed which can exploit the attributes of visual attention information adequately. The experimental results with respect to the subjective measurement demonstrate that the proposed metric outperforms the current methods.

A model for measuring Quality of Experience

In this paper we present a model for measuring the quality of experience (QoE) of multimedia services. Our model consists of measurable and non-measurable parameters and describes a framework for quantifying the model parameters by transformation into an XML document with discreet values. The model has been validated by running a field trial over 4 weeks using 3G mobile phones for a video on demand service and mobile TV

Universal multimedia access from wired and wireless systems

Personal computing and communication devices such as computers, personal digital assistants (PDAs), and mobile phones are moving to their next generation in which the end user will be able to access a multitude of information with a single device either locally or through a network. One likely trend in future personal computing and personal communication is that there will not be a single but several equivalent devices available to users allowing access to information in various forms. Each user, depending on his/her needs would access one or several among them depending on the situation and his/her preference. Using existing protocol mechanisms, in this case, a mapping and negotiation of resources during connection setup would be performed, which would remain in place throughout the life of the connection.This paper provides an overview of universal multimedia access (UMA), a concept for accessing multimedia content through a variety of possible schemes, and discusses some of the issues that arise regarding its deployment. In particular, UMA will provide a solution for adapting the delivered content when users attempt to access their choice irrespective of their terminal characteristics and communication infrastructure, as apposed to the assumption that the content remains fixed and the objective is to deliver the original content at all times. This recognition represents the impetus for the development of media descriptions and hence UMA; that is, the notion that valuable information can be derived from a variety of conversions of a multimedia content source.The issues discussed are future requirements on content servers and multimedia viewers, media conversions, UMA protocols, and UMA network architectures. The problems addressed are quality of service issues in network solutions for multimedia communications and reconfigurable architectures and network control based on source adaptations through media conversions and transcoding.

Adaptive H.264/MPEG-4 SVC video over IEEE 802.16 broadband wireless networks

In this paper we present a solution for delivering streaming video-on-demand to subscribers via an 802.16 broadband wireless access network. The solution leverages the forthcoming H.264/AVC Scalable Video Coding (SVC) scheme and a mechanism to perform rate adaptation based on monitoring changes to the amount of flow traffic in the network at any time. A simulation-based approach is used to determine how the system performs in a rural deployment. Results show that the scheme provides high utilization of the wireless access system, at over 96%. Further, it maintains smooth transmission rate for the video applications, and ensures that no interruptions in continuous video playback occur during the streaming session. Lastly, a comparison with single-layer H.264/AVC is performed, showing how the proposed solution performs better with respect to both system utilization and the fact that no clients suffer interruptions in continuous playback.

An HVS-based no-reference perceptual quality assessment of JPEG coded images using neural networks

In this paper, we present a novel no-reference (NR) metric to assess the quality of JPEG-coded images. The features for predicting the perceived image quality are extracted by considering the key human visual sensitivity factors such as, edge amplitude, edge length, background activity and background luminance. The extracted features with the subjective test results are used to train a multi-layer perceptron (MLP) neural network. Experimental results show that the prediction of the trained neural network is very close to the mean opinion score (MOS). The subjective test results of the proposed metric are compared with the Wang-Boviks NR blockiness metric. Further, this metric can be extended to assess the quality of the MPLG/H.26x compressed videos.

Assessment of Stereoscopic Crosstalk Perception

Stereoscopic three-dimensional (3-D) services do not always prevail when compared with their two-dimensional (2-D) counterparts, though the former can provide more immersive experience with the help of binocular depth. Various specific 3-D artefacts might cause discomfort and severely degrade the Quality of Experience (QoE). In this paper, we analyze one of the most annoying artefacts in the visualization stage of stereoscopic imaging, namely, crosstalk, by conducting extensive subjective quality tests. A statistical analysis of the subjective scores reveals that both scene content and camera baseline have significant impacts on crosstalk perception, in addition to the crosstalk level itself. Based on the observed visual variations during changes in significant factors, three perceptual attributes of crosstalk are summarized as the sensorial results of the human visual system (HVS). These are shadow degree, separation distance, and spatial position of crosstalk. They are classified into two categories: 2-D and 3-D perceptual attributes, which can be described by a Structural SIMilarity (SSIM) map and a filtered depth map, respectively. An objective quality metric for predicting crosstalk perception is then proposed by combining the two maps. The experimental results demonstrate that the proposed metric has a high correlation (over 88%) when compared with subjective quality scores in a wide variety of situations.

Attention modeling for video quality assessment: Balancing global quality and local quality

This paper proposes to evaluate video quality by balancing two quality components: global quality and local quality. The global quality is a result from subjects allocating their attention equally to all regions in a frame and all frames in a video. It is evaluated by image quality metrics (IQM) with averaged spatiotemporal pooling. The local quality is derived from visual attention modeling and quality variations over frames. Saliency, motion, and contrast information are taken into account in modeling visual attention, which is then integrated into IQMs to calculate the local quality of a video frame. The local quality of a video sequence is calculated by pooling local quality values over all frames with a temporal pooling scheme derived from the known relationship between perceived video quality and the frequency of temporal quality variations. The overall quality of a distorted video is a weighted average between the global quality and the local quality. Experimental results demonstrate that the combination of the global quality and local quality outperforms both sole global quality and local quality, as well as other quality models, in video quality assessment. In addition, the proposed video quality modeling algorithm can improve the performance of image quality metrics on video quality assessment compared to the normal averaged spatiotemporal pooling scheme.

Attention Driven Foveated Video Quality Assessment

Contrast sensitivity of the human visual system to visual stimuli can be significantly affected by several mechanisms, e.g., vision foveation and attention. Existing studies on foveation based video quality assessment only take into account static foveation mechanism. This paper first proposes an advanced foveal imaging model to generate the perceived representation of video by integrating visual attention into the foveation mechanism. For accurately simulating the dynamic foveation mechanism, a novel approach to predict video fixations is proposed by mimicking the essential functionality of eye movement. Consequently, an advanced contrast sensitivity function, derived from the attention driven foveation mechanism, is modeled and then integrated into a wavelet-based distortion visibility measure to build a full reference attention driven foveated video quality (AFViQ) metric. AFViQ exploits adequately perceptual visual mechanisms in video quality assessment. Extensive evaluation results with respect to several publicly available eye-tracking and video quality databases demonstrate promising performance of the proposed video attention model, fixation prediction approach, and quality metric.