Tuomas Leisti

Measuring Stereoscopic Image Quality Experience with Interpretation Based Quality Methodology

Stereoscopic technologies have developed significantly in recent years. These advances require also more understanding of the experiental dimensions of stereoscopic contents. In this article we describe experiments in which we explore the experiences that viewers have when they view stereoscopic contents. We used eight different contents that were shown to the participants in a paired comparison experiment where the task of the participants was to compare the same content in stereoscopic and non-stereoscopic form. The participants indicated their preference but were also interviewed about the arguments they used when making the decision. By conducting a qualitative analysis of the interview texts we categorized the significant experiental factors related to viewing stereoscopic material. Our results indicate that reality-likeness as well as artificiality were often used as arguments in comparing the stereoscopic materials. Also, there were more emotional terms in the descriptions of the stereoscopic films, which might indicate that the stereoscopic projection technique enhances the emotions conveyed by the film material. Finally, the participants indicated that the three-dimensional material required longer presentation time, as there were more interesting details to see.

Content and quality: Interpretation-based estimation of image quality

Test image contents affect subjective image-quality evaluations. Psychometric methods might show that contents have an influence on image quality, but they do not tell what this influence is like, i.e., how the contents influence image quality. To obtain a holistic description of subjective image quality, we have used an interpretation-based quality (IBQ) estimation approach, which combines qualitative and quantitative methodology. The method enables simultaneous examination of psychometric results and the subjective meanings related to the perceived image-quality changes. In this way, the relationship between subjective feature detection, subjective preferences, and interpretations are revealed. We report a study that shows that different impressions are conveyed in five test image contents after similar sharpness variations. Thirty naïve observers classified and freely described the images after which magnitude estimation was used to verify that they distinguished the changes in the images. The data suggest that in the case of high image quality, the test image selection is crucial. If subjective evaluation is limited only to technical defects in test images, important subjective information of image-quality experience is lost. The approach described here can be used to examine image quality and it will help image scientists to evaluate their test images.

Evaluating the multivariate visual quality performance of image-processing components

The estimation of image quality is a demanding task, especially when estimating different high-quality imaging products or their components. The challenge is the multivariate nature of image quality as well as the need to use naïve observers as test subjects, since they are the actual end-users of the products. Here, we use a subjective approach suitable for estimating the quality performance of different imaging device components with naïve observers—the interpretation-based quality (IBQ) approach. From two studies with 61 naïve observers, 17 natural image contents, and 13 different camera image signal processor pipelines, we determined the subjectively crucial image quality attributes and dimensions and the description of each pipelines perceived image quality performance. We found that the subjectively most important image quality dimensions were color shift/naturalness, darkness, and sharpness. The first dimension, which was related to naturalness and colors, distinguished the good-quality pipelines from the middle- and low-quality groups, and the dimensions of darkness and sharpness described why the quality failed in the low-quality pipelines. The study suggests that the high-level concept naturalness is a requirement for high-quality images, whereas quality can fail for other reasons in low-quality images, and this failure can be described by low-level concepts, such as darkness and sharpness.

What do users really perceive - : probing the subjective image quality

Image evaluation schemes must fulfill both objective and subjective requirements. Objective image quality evaluation models are often preferred over subjective quality evaluation, because of their fastness and cost-effectiveness. However, the correlation between subjective and objective estimations is often poor. One of the key reasons for this is that it is not known what image features subjects use when they evaluate image quality. We have studied subjective image quality evaluation in the case of image sharpness. We used an Interpretation-based Quality (IBQ) approach, which combines both qualitative and quantitative approaches to probe the observers quality experience. Here we examine how naive subjects experienced and classified natural images, whose sharpness was changing. Together the psychometric and qualitative information obtained allows the correlation of quantitative evaluation data with its underlying subjective attribute sets. This offers guidelines to product designers and developers who are responsible for image quality. Combining these methods makes the end-user experience approachable and offers new ways to improve objective image quality evaluation schemes.

Subjective experience of image quality: attributes, definitions, and decision making of subjective image quality

Subjective quality rating does not reflect the properties of the image directly, but it is the outcome of a quality decision making process, which includes quantification of subjective quality experience. Such a rich subjective content is often ignored. We conducted two experiments (with 28 and 20 observers), in order to study the effect of paper grade on image quality experience of the ink-jet prints. Image quality experience was studied using a grouping task and a quality rating task. Both tasks included an interview, but in the latter task we examined the relations of different subjective attributes in this experience. We found out that the observers use an attribute hierarchy, where the high-level attributes are more experiential, general and abstract, while low-level attributes are more detailed and concrete. This may reflect the hierarchy of the human visual system. We also noticed that while the observers show variable subjective criteria for IQ, the reliability of average subjective estimates is high: when two different observer groups estimated the same images in the two experiments, correlations between the mean ratings were between .986 and .994, depending on the image content.

Bayesian network model of overall print quality: Construction and structural optimisation

Prediction of overall visual quality based on instrumental measurements is a challenging task. Despite the several proposed models and methods, there exists a gap between the instrumental measurements of print and human visual assessment of natural images. In this work, a computational model for representing and quantifying the overall visual quality of prints is proposed. The computed overall quality should correspond to the human visual quality perception when viewing the printed images. The proposed model is a Bayesian network which connects the objective instrumental measurements to the subjective opinion distribution of human observers. This relationship can be used to score printed images, and additionally, to computationally study the connections of the attributes. A novel graphical learning approach using an iterative evolve-estimate-simulate loop learning the quality model based on psychometric data and instrumental measurements is suggested. The network structure is optimised by applying evolutionary computation (evolve). The estimation of the Bayesian network parameters is within the evolutionary loop. In this loop, the maximum likelihood approach is used (estimate). The stochastic learning process is guided by priors devised from the psychometric subjective experiments (performance through simulation). The model reveals and represents the explanatory factors between its elements providing insight to the psychophysical phenomenon of how observers perceive visual quality and which measurable entities affect the quality perception. By using true data, the design choices are demonstrated. It is also shown that the best-performing network establishes a clear and intuitively correct structure between the objective measurements and psychometric data.

Is there hope for predicting human visual quality experience

One of the most important research goals in media science is a computational model for the human perception of visual quality, that is, how to predict the subjective visual quality experience. This research area has converged to developing new and investigating existing lower-level measurable quantities, physical, visual or computational, which could explain the high level experience. A principal research question, whether the prediction of the visual quality experience based on any lower-level objective measurements is possible at all, has received much less attention. This question is investigated in this study. First, we describe a large psychological experiment where true factors of the human quality experience are pair-wise resolved for dedicatedly selected samples. Second, we describe a ranking measure which reveals the relationship between selected measurable quantities and the human evaluation. Finally, the presented ranking method is used to provide quantitative evidence that visual quality experience can be predicted using lower-level measurable quantities. This result is novel and by simultaneously revealing the underlying lower-level factors it should re-direct the future research towards the true model.

Finding best measurable quantities for predicting human visual quality experience

The literature of visual quality is mainly concentrated on devising new physical, visual, or computational quality features which could indirectly reflect ldquotrue visual qualityrdquo. The problem is that the true visual quality is always a subjective and context sensitive judgement of a single individual or a group of individuals. Therefore, the developed methods are only loosely connected to this ultimate objective, and the existing de facto and official standards have been designed by forming a consensus among experts of a specific field (e.g., in the printing industry). In this study, we describe a large psychological experiment where true factors of the human quality experience are pair-wise resolved for dedicatedly selected samples. Then we describe a ranking measure which reveals the relationship between selected measurable quantities and the human evaluation trial. Finally by using the above framework, we devise the best combinations from a set of well-known measurable quantities. The devised combinations can be considered as optimal when agreement with the human visual quality experience is desired, and therefore, they also reveal completely novel information about measuring visual quality.

Framework for modeling visual printed image quality from the paper perspective

Due to the rise in performance of digital printing, image-based applications are gaining popularity. This creates needs for specifying the quality potential of printers and materials in more detail than before. Both production and end-use standpoints are relevant. This paper gives an overview of an on-going study which has the goal of determining a framework model for the visual quality potential of paper in color image printing. The approach is top-down and it is founded on the concept of a layered network model. The model and its subjective, objective and instrumental measurement layers are discussed. Some preliminary findings are presented. These are based on data from samples obtained by printing natural image contents and simple test fields on a wide range of paper grades by ink-jet in a color managed process. Color profiles were paper specific. Visual mean opinion score data by human observers could be accounted for by two or three dimensions. In the first place these are related to brightness and color brightness. Image content has a marked effect on the dimensions. This underlines the challenges in designing the test images.

Evaluation of the visual performance of image processing pipes: information value of subjective image attributes

Subjective image quality data for 9 image processing pipes and 8 image contents (taken with mobile phone camera, 72 natural scene test images altogether) from 14 test subjects were collected. A triplet comparison setup and a hybrid qualitative/quantitative methodology were applied. MOS data and spontaneous, subjective image quality attributes to each test image were recorded. The use of positive and negative image quality attributes by the experimental subjects suggested a significant difference between the subjective spaces of low and high image quality. The robustness of the attribute data was shown by correlating DMOS data of the test images against their corresponding, average subjective attribute vector length data. The findings demonstrate the information value of spontaneous, subjective image quality attributes in evaluating image quality at variable quality levels. We discuss the implications of these findings for the development of sensitive performance measures and methods in profiling image processing systems and their components, especially at high image quality levels.