Hojatollah Yeganeh

Objective Quality Assessment of Tone-Mapped Images

Tone-mapping operators (TMOs) that convert high dynamic range (HDR) to low dynamic range (LDR) images provide practically useful tools for the visualization of HDR images on standard LDR displays. Different TMOs create different tone-mapped images, and a natural question is which one has the best quality. Without an appropriate quality measure, different TMOs cannot be compared, and further improvement is directionless. Subjective rating may be a reliable evaluation method, but it is expensive and time consuming, and more importantly, is difficult to be embedded into optimization frameworks. Here we propose an objective quality assessment algorithm for tone-mapped images by combining: 1) a multiscale signal fidelity measure on the basis of a modified structural similarity index and 2) a naturalness measure on the basis of intensity statistics of natural images. Validations using independent subject-rated image databases show good correlations between subjective ranking score and the proposed tone-mapped image quality index (TMQI). Furthermore, we demonstrate the extended applications of TMQI using two examples - parameter tuning for TMOs and adaptive fusion of multiple tone-mapped images.

High Dynamic Range Image Compression by Optimizing Tone Mapped Image Quality Index

Tone mapping operators (TMOs) aim to compress high dynamic range (HDR) images to low dynamic range (LDR) ones so as to visualize HDR images on standard displays. Most existing TMOs were demonstrated on specific examples without being thoroughly evaluated using well-designed and subject-validated image quality assessment models. A recently proposed tone mapped image quality index (TMQI) made one of the first attempts on objective quality assessment of tone mapped images. Here, we propose a substantially different approach to design TMO. Instead of using any predefined systematic computational structure for tone mapping (such as analytic image transformations and/or explicit contrast/edge enhancement), we directly navigate in the space of all images, searching for the image that optimizes an improved TMQI. In particular, we first improve the two building blocks in TMQI-structural fidelity and statistical naturalness components-leading to a TMQI-II metric. We then propose an iterative algorithm that alternatively improves the structural fidelity and statistical naturalness of the resulting image. Numerical and subjective experiments demonstrate that the proposed algorithm consistently produces better quality tone mapped images even when the initial images of the iteration are created by the most competitive TMOs. Meanwhile, these results also validate the superiority of TMQI-II over TMQI.

A Patch-Structure Representation Method for Quality Assessment of Contrast Changed Images

Contrast is a fundamental attribute of images that plays an important role in human visual perception of image quality. With numerous approaches proposed to enhance image contrast, much less work has been dedicated to automatic quality assessment of contrast changed images. Existing approaches rely on global statistics to estimate contrast quality. Here we propose a novel local patch-based objective quality assessment method using an adaptive representation of local patch structure, which allows us to decompose any image patch into its mean intensity, signal strength and signal structure components and then evaluate their perceptual distortions in different ways. A unique feature that differentiates the proposed method from previous contrast quality models is the capability to produce a local contrast quality map, which predicts local quality variations over space and may be employed to guide contrast enhancement algorithms. Validations based on four publicly available databases show that the proposed patch-based contrast quality index (PCQI) method provides accurate predictions on the human perception of contrast variations.

Objective assessment of tone mapping algorithms

There has been a growing interest in recent years to develop tone mapping algorithms that can convert high dynamic range (HDR) to low dynamic range (LDR) images, so that they can be visualized on standard displays. With a number of tone mapping algorithms proposed, a natural question is which one gives the best performance. Although subjective assessment methods provide useful references, they are expensive and time-consuming, and are difficult to be embedded into the design stage of tone mapping algorithms for optimization and parameter tuning purposes. This paper focuses on objective assessment of tone mapping operators. Inspired by the success of the structural similarity index method for image quality assessment, we propose a new objective assessment algorithm that creates multi-scale similarity maps between HDR and LDR images. Our experiments show that the proposed method correlates well with subjective rankings of existing tone mapping operators. Furthermore, we demonstrate how the proposed algorithm can be employed in an existing tone mapping algorithm for optimal parameter tuning.

Study of the effects of stalling events on the quality of experience of mobile streaming videos

We have created a new mobile video database that models distortions caused by network impairments. In particular, we simulate stalling events and startup delays in over-the-top (OTT) mobile streaming videos. We describe the way we simulated diverse stalling events to create a corpus of distorted videos and the human study we conducted to obtain subjective scores. We also analyzed the ratings to understand the impact of several factors that influence the quality of experience (QoE). To the best of our knowledge, ours is the most comprehensive and diverse study on the effects of stalling events on QoE. We are making the database publicly available [1] in order to help advance state-of-the-art research on user-centric mobile network planning and management.

Objective quality assessment for image super-resolution: A natural scene statistics approach

There has been an increasing number of image super-resolution (SR) algorithms proposed recently to create images with higher spatial resolution from low-resolution (LR) images. Nevertheless, how to evaluate the performance of such SR and interpolation algorithms remains an open problem. Subjective assessment methods are useful and reliable, but are expensive, time-consuming, and difficult to be embedded into the design and optimization procedures of SR and interpolation algorithms. Here we make one of the first attempts to develop an objective quality assessment method of a given resolution-enhanced image using the available LR image as a reference. Our algorithm follows the philosophy behind the natural scene statistics (NSS) approach. Specifically, we build statistical models of frequency energy falloff and spatial continuity based on high quality natural images and use the departures from such models to quantify image quality degradations. Subjective experiments have been carried out that verify the effectiveness of the proposed approach.

High dynamic range image tone mapping by optimizing tone mapped image quality index

An active research topic in recent years is to design tone mapping operators (TMOs) that convert high dynamic range (H-DR) to low dynamic range (LDR) images, so that HDR images can be visualized on standard displays. Nevertheless, most existing work has been done in the absence of a well-established and subject-validated image quality assessment (IQA) model, without which fair comparisons and further improvement are difficult. Recently, a tone mapped image quality index (TMQI) was proposed, which has shown to have good correlation with subjective evaluations of tone mapped images. Here we propose a substantially different approach to design TMO, where instead of using any pre-defined systematic computational structure (such as image transformation or contrast/edge enhancement) for tone mapping, we navigate in the space of all images, searching for the image that optimizes TMQI. The navigation involves an iterative process that alternately improves the structural fidelity and statistical naturalness of the resulting image, which are the two fundamental building blocks in TMQI. Experiments demonstrate the superior performance of the proposed method.

Structural fidelity vs. naturalness - objective assessment of tone mapped images

There has been an increasing number of tone mapping algorithms developed in recent years that can convert high dynamic range (HDR) to low dynamic range (LDR) images, so that they can be visualized on standard displays. Nevertheless, good quality evaluation criteria of tone mapped images are still lacking, without which, different tone mapping algorithms cannot be compared and there is no meaningful direction for improvement. Although subjective assessment methods provide useful references, they are expensive and time-consuming, and are difficult to be embedded into optimization frameworks. In this paper, we propose a novel objective assessment method that combines a multiscale signal fidelity measure inspired by the structural similarity (SSIM) index and a naturalness measure based on statistics on the brightness of natural images. Validations using available subjective data show good correlations between the proposed measure and subjective rankings of LDR images created by existing tone mapping operators.

Delivery quality score model for Internet video

The vast majority of todays internet video services are consumed over-the-top (OTT) via reliable streaming (HTTP via TCP), where the primary noticeable delivery-related impairments are startup delay and stalling. In this paper we introduce an objective model called the delivery quality score (DQS) model, to predict users QoE in the presence of such impairments. We describe a large subjective study that we carried out to tune and validate this model. Our experiments demonstrate that the DQS model correlates highly with the subjective data and that it outperforms other emerging models.

High dynamic range image tone mapping by maximizing a structural fidelity measure

Tone mapping operators (TMOs) that convert high dynamic range (HDR) images to standard low dynamic range (LDR) images are highly desirable for the visualization of these images on standard displays. Although many existing TMOs produce visually appealing images, it is until recently validated objective measures that can assess their quality have been proposed. Without such objective measures, the design of traditional TMOs can only be based on intuitive ideas, lacking clear goals for further improvement. In this paper, we propose a substantially different tone mapping approach, where instead of explicitly designing a new computational structure for TMO, we search in the space of images to find better quality images in terms of a recent objective measure that can assess the structural fidelity between two images of different dynamic ranges. Specifically, starting from any initial image, the proposed algorithm moves the image along the gradient ascent direction and stops until it converges to a maximal point. Our experiments show that the proposed algorithm reliably produces better quality images upon a number of state-of-the-art TMOs.