Lawrence K. Cormack

Study of Subjective and Objective Quality Assessment of Video

We present the results of a recent large-scale subjective study of video quality on a collection of videos distorted by a variety of application-relevant processes. Methods to assess the visual quality of digital videos as perceived by human observers are becoming increasingly important, due to the large number of applications that target humans as the end users of video. Owing to the many approaches to video quality assessment (VQA) that are being developed, there is a need for a diverse independent public database of distorted videos and subjective scores that is freely available. The resulting Laboratory for Image and Video Engineering (LIVE) Video Quality Database contains 150 distorted videos (obtained from ten uncompressed reference videos of natural scenes) that were created using four different commonly encountered distortion types. Each video was assessed by 38 human subjects, and the difference mean opinion scores (DMOS) were recorded. We also evaluated the performance of several state-of-the-art, publicly available full-reference VQA algorithms on the new database. A statistical evaluation of the relative performance of these algorithms is also presented. The database has a dedicated web presence that will be maintained as long as it remains relevant and the data is available online.

Full-reference quality assessment of stereopairs accounting for rivalry

We develop a framework for assessing the quality of stereoscopic images that have been afflicted by possibly asymmetric distortions. An intermediate image is generated which when viewed stereoscopically is designed to have a perceived quality close to that of the cyclopean image. We hypothesize that performing stereoscopic QA on the intermediate image yields higher correlations with human subjective judgments. The experimental results confirm the hypothesis and show that the proposed framework significantly outperforms conventional 2D QA metrics when predicting the quality of stereoscopically viewed images that may have been asymmetrically distorted.

A subjective study to evaluate video quality assessment algorithms

Automatic methods to evaluate the perceptual quality of a digital video sequence have widespread applications wherever the end-user is a human. Several objective video quality assessment (VQA) algorithms exist, whose performance is typically evaluated using the results of a subjective study performed by the video quality experts group (VQEG) in 2000. There is a great need for a free, publicly available subjective study of video quality that embodies state-of-the-art in video processing technology and that is effective in challenging and benchmarking objective VQA algorithms. In this paper, we present a study and a resulting database, known as the LIVE Video Quality Database, where 150 distorted video sequences obtained from 10 different source video content were subjectively evaluated by 38 human observers. Our study includes videos that have been compressed by MPEG-2 and H.264, as well as videos obtained by simulated transmission of H.264 compressed streams through error prone IP and wireless networks. The subjective evaluation was performed using a single stimulus paradigm with hidden reference removal, where the observers were asked to provide their opinion of video quality on a continuous scale. We also present the performance of several freely available objective, full reference (FR) VQA algorithms on the LIVE Video Quality Database. The recent MOtion-based Video Integrity Evaluation (MOVIE) index emerges as the leading objective VQA algorithm in our study, while the performance of the Video Quality Metric (VQM) and the Multi-Scale Structural SIMilarity (MS-SSIM) index is noteworthy. The LIVE Video Quality Database is freely available for download1 and we hope that our study provides researchers with a valuable tool to benchmark and improve the performance of objective VQA algorithms.

No-Reference Quality Assessment of Natural Stereopairs

We develop a no-reference binocular image quality assessment model that operates on static stereoscopic images. The model deploys 2D and 3D features extracted from stereopairs to assess the perceptual quality they present when viewed stereoscopically. Both symmetric- and asymmetric-distorted stereopairs are handled by accounting for binocular rivalry using a classic linear rivalry model. The NSS features are used to train a support vector machine model to predict the quality of a tested stereopair. The model is tested on the LIVE 3D Image Quality Database, which includes both symmetric- and asymmetric-distorted stereoscopic 3D images. The experimental results show that our proposed model significantly outperforms the conventional 2D full-reference QA algorithms applied to stereopairs, as well as the 3D full-reference IQA algorithms on asymmetrically distorted stereopairs.

Interocular correlation, luminance contrast and cyclopean processing

We have investigated the nature and viability of interocular correlation as a measure of signal strength in the cyclopean domain. Thresholds for the detection of interocular correlation in dynamic random element stereograms were measured as a function of luminance contrast, a more traditional measure of stimulus strength. At high contrasts, correlation thresholds were independent of contrast. At low contrasts, correlation thresholds were inversely proportional to the square of contrast. Stereothresholds were also measured as a function of both contrast and interocular correlation. At low contrasts, stereoacuity was inversely proportional to both interocular correlation and the square of contrast. These results are consistent with an inherently multiplicative mechanism of binocular combination, such as a cross-correlation of the two eyes inputs.

Negative feedback control model of proximal convergence and accommodation

A comprehensive model has been developed to illustrate the interactions between the observer and the surrounding environment in the control of oculomotor responses to distance or 3‐D space. Accommodation and vergence respond to both spatiotopic (body referenced) proximal percepts and retinotopic (eye referenced) physical stimuli of blur and disparity. Both spatiotopic and retinotopic stimuli are derived respectively from perceptual and physical correlates of negative feedback for eye position. The spatiotopic and retinotopic stimulus errors are combined in the feed forward path and drive a common occulomotor controller which has a phasic‐tonic organization. Spatiotopic and retinotopic stimuli are shown to be effective over complementary operating ranges. Perceptual spatiotopic errors of gaze provide optimal stimuli for near responses to large depth intervals whereas physical‐retinotopic cues of blur and disparity provide quantitative information about small binocular fixation errors. Small dynamic variations of target distance are sensed both spatiotopically and retinotopically. Coarse and fine spatiotopic errors of gaze are processed differently. Large spatiotopic errors are sampled intermittently al the beginning of the near response, whereas small retinotopic position errors and spatiotopic velocity errors are sampled continuously throughout the near response. Former reports of empirically observed higher velocity of vergence responses to very large depth intervals is explained in terms of stimulus sampling modes rather than in terms of separate oculomotor control mechanisms. The model demonstrates a complementary function of top‐down spatiotopic cues, which are used to initiate the near response, and bottom‐up retinotopic cues, which are used to refine and complete the near response. Cross‐couplings by vergence‐accommodation and accommodative‐vergence serve to coordinate the components of the near response when feedback from the sensed response of one motor system (i.e. vergence) is more accurate than that of the other motor system (i.e. accommodation). The model presented here is concerned primarily with the near response mediated by accommodation and disjunctive eye movements and not by the independent vergence mediated by non‐conjugate or yoked saccades of unequal amplitude.

Disparity- and velocity-based signals for three-dimensional motion perception in human MT+

How does the primate visual system encode three-dimensional motion? The macaque middle temporal area (MT) and the human MT complex (MT+) have well-established sensitivity to two-dimensional frontoparallel motion and static disparity. However, evidence for sensitivity to three-dimensional motion has remained elusive. We found that human MT+ encodes two binocular cues to three-dimensional motion: changing disparities over time and interocular comparisons of retinal velocities. By varying important properties of moving dot displays, we distinguished these three-dimensional motion signals from their constituents, instantaneous binocular disparity and monocular retinal motion. An adaptation experiment confirmed direction selectivity for three-dimensional motion. Our results indicate that MT+ carries critical binocular signals for three-dimensional motion processing, revealing an important and previously overlooked role for this well-studied brain area.

Disparity tuning in mechanisms of human stereopsis

The change in sensitivity across some stimulus dimension which follows adaptation to a particular stimulus can reveal a great deal about the tuning characteristics of underlying sensory/perceptual mechanisms. In this study, a psychophysical adaptation paradigm was employed to characterize the disparity tuning of perceptual mechanisms involved in stereopsis. The stimulus was a dynamic random-dot stereogram (DRDS) portraying a surface which varied in interocular correlation (IOC) and retinal disparity. Adaptation to a fully correlated DRDS surface produced an elevation in IOC threshold over a relatively narrow range of disparities, with maximum effect at the disparity of the adapting stimulus. The width of these disparity tuning functions varied from 5 arc min for adaptation at the horopter to 20 arc min for adaptation at 20 arc min disparity. Frequently, IOC sensitivity was enhanced for disparities on either side of the adapted disparity, suggesting that an opponent center-surround organization operates at an early level of disparity processing. A model of underlying channel structure consistent with these data is presented.

DOVES: a database of visual eye movements.

DOVES, a database of visual eye movements, is a set of eye movements collected from 29 human observers as they viewed 101 natural calibrated images. Recorded using a high-precision dual-Purkinje eye tracker, the database consists of around 30 000 fixation points, and is believed to be the first large-scale database of eye movements to be made available to the vision research community. The database, along with MATLAB functions for its use, may be downloaded freely from http://live.ece.utexas.edu/research/doves, and used without restriction for educational and research purposes, providing that this paper is cited in any published work. This paper documents the acquisition procedure, summarises common eye movement statistics, and highlights numerous research topics for which DOVES may be used.

Evolved navigation theory and the descent illusion.

Researchers often assume that height perception results from all of the same mechanisms as does other distance perception (Avraamides, Loomis, Klatzky, & Golledge, 2004; Foley, Ribeiro-Filho, & Da Silva, 2004; Wu, Ooi, & He, 2004). Evolved navigation theory (ENT) proposes that natural selection has differentiated some psychological processes, including height perception, in response to the navigational outcome of falling. We tested predictions from three theories in two experiments. Only ENT predicted greater height perceived from the top than from the bottom of a vertical surface (because descent results in falls more often than does ascent). Participants across experiments perceived an average of 32% greater vertical distance when viewing from the top than when viewing from the bottom. We discuss selected implications and suggest ENT for uniting isolated findings, including the vertical-horizontal illusion.