Lew B. Stelmach

Stereo image quality: effects of mixed spatio-temporal resolution

We explored the response of the human visual system to mixed-resolution stereo video-sequences, in which one eye view was spatially or temporally low-pass filtered. It was expected that the perceived quality, depth, and sharpness would be relatively unaffected by low-pass filtering, compared to the case where both eyes viewed a filtered image. Subjects viewed two 10-second stereo video-sequences, in which the right-eye frames were filtered vertically (V) and horizontally (H) at 1/2 H, 1/2 V, 1/4 H, 1/4 V, 1/2 H 1/2 V, 1/2 H 1/4 V, 1/4 H 1/2 V, and 1/4 H 1/4 V resolution. Temporal filtering was implemented for a subset of these conditions at 1/2 temporal resolution, or with drop-and-repeat frames. Subjects rated the overall quality, sharpness, and overall sensation of depth. It was found that spatial filtering produced acceptable results: the overall sensation of depth was unaffected by low-pass filtering, while ratings of quality and of sharpness were strongly weighted towards the eye with the greater spatial resolution. By comparison, temporal filtering produced unacceptable results: field averaging and drop-and-repeat frame conditions yielded images with poor quality and sharpness, even though perceived depth was relatively unaffected. We conclude that spatial filtering of one channel of a stereo video-sequence may be an effective means of reducing the transmission bandwidth.

Unequal weighting of monocular inputs in binocular combination: implications for the compression of stereoscopic imagery

For efficient storage and transmission of stereoscopic images over bandwidth-limited channels, compression can be achieved by degrading 1 monocular input of a stereo pair and maintaining the other at the desired quality. The desired quality of the fused stereoscopic image can be achieved, provided that binocular vision assigns greater weight to the nondegraded input. A psychophysical matching procedure was used to determine if such over-weighting occurred when the monocular degradation included blur or blocking artifacts. Over-weighting of the nondegraded input occurred for blur, but under-weighting of the nondegraded input occurred for blockiness. Some participants exhibited ocular dominance, but this did not affect the blur results. The authors conclude that blur, but not blockiness, is an acceptable form of monocular degradation.

Human perception of mismatched stereoscopic 3D inputs

The bandwidth required to transmit stereoscopic video images is nominally twice that required for standard, monoscopic images. One method of reducing the required bandwidth is to code the two video streams asymmetrically. We assessed the perceptual impact of this bandwidth-reduction technique for low-pass filtering, DCT-based quantization, and a combination of filtering and quantization. It was found that the binocular percept depended on the type of degradation: for low-pass filtering, the binocular percept was dominated by the high-quality image, whereas for quantization it corresponded to the average of the inputs to the two eyes. The results indicated that asymmetrical coding is a promising technique for reducing storage and transmission bandwidth of stereoscopic sequences.

Improving the visual comfort of stereoscopic images

We compared the visual comfort and apparent depth of stereoscopic images for three camera configurations: parallel (without image shift), image-shifted and converged. In the parallel and image-shifted configurations, the stereo cameras were pointed straight ahead. In the converged configuration the cameras were toed-in. In the image-shifted configuration the image frame was shifted perpendicularly with respect to the line of sight of the camera. The parallel configuration produces images with uncomfortably large disparities for objects near the camera. By converging the cameras or by shifting the image, these large disparities can be reduced and visual comfort can be improved. However, the converged configuration introduces keystone distortions into the image, which can produce visual discomfort. The image-shifted configuration does not introduce keystone distortions, but affects the width of the image frame. It also requires unusual camera hardware or computer post-processing to shift the images. We found that converged and image-shifted configurations improved the visual comfort of stereoscopic images by an equivalent amount, without affecting the apparent depth. Keystone distortions in the converged configuration had no appreciable negative effect on visual comfort.

Perceived Smoothness of Viewpoint Transition in Multi-Viewpoint Stereoscopic Displays

Three experiments were conducted to investigate the perceived smoothness of viewpoint transition in multi-view images. Different viewpoints of a stereoscopic scene were generated in real-time. The left-eye and right-eye views of each viewpoint were viewed stereoscopically, from a distance of 120 cm, with shutter glasses synchronized to the display. In Experiment 1, different vantage points of the scene were displayed as the viewer moved his/her head left and right in front of the display. Viewers rated the perceived smoothness of the scene for different viewpoint densities, i.e., number of viewpoints displayed per unit of amplitude of lateral movement, and extent of look-around, i.e., angular separation between the leftmost and rightmost rendered viewpoints. The second and third experiments were similar to the first with the exception that the change in displayed viewpoint was either controlled by the viewer’s hand (Experiment 2) or occurred without any intervention on the part of the viewer (Experiment 3). Perceived smoothness improved with increasing viewpoint density in all three experiments. Perceived smoothness of viewpoint transition was affected by the extent of look-around, but the effect varied across experiments.

Bandwidth reduction for stereoscopic video signals

The bandwidth required to transmit stereoscopic video signals is nominally twice that required for standard, monoscopic images. An effective method of reducing the required bandwidth is to code the two video streams asymmetrically. We assessed the impact of this bandwidth- reduction technique on image quality and overall sensation of depth. Images from the right-eye stream were spatially filtered on image quality and overall sensation of depth. Images from the right-eye stream were stream were spatially filtered to half and quarter resolution. Subsequently, the images were processed using an MPEG-2 codec at bit-rates of 6, 2, and 1 Mbit/s. Subjects assessed image quality and depth using a double-stimulus, continuous-quality scale method. It was found that perceived depth was relatively robust to spatial filtering and bit-rate reduction. Image quality was affected more by bit-rate reduction than by spatial filtering and, at the lower bit rates, ratings were much higher for stereoscopic than for non-stereoscopic sequences. The results indicate that asymmetrical coding of stereoscopic sequences can be an effective means of reducing bandwidth for storage and transmission.

Stereo-anomalous vision in a sample of young adults

We estimated the proportion of people that have defective stereo vision and are unable to utilize stereo disparity information to perceive depth. Previous estimates have ranged anywhere from as high as 30% to as low as 6%. Our goal was to understand the basis for the wide range in these estimates. To do this, we administered two psychophysical tests to a sample of 100 young adults. Visual stimuli consisted of dynamic random-dot stereograms presented using a fast-decay, time-sequential display device. The stimuli covered a range of disparities between 0 and .38 degrees (both crossed and uncrossed). A forced-choice methodology was used to estimate whether subjects could perceive depth based on horizontal disparity. It was found that display duration was a key variable determining the number of viewers that were classified as stereo-anomalous. The relatively high incidence of stereo- anomalous viewers in previous research was explained by the short display durations (80 ms) used in those studies. With longer durations of about 1 sec, we found that only about 5% of viewers had defective stereo vision.

Investigation on the effects of disparity-based asymmetrical filtering of stereoscopic video

Current binocular stereoscopic displays cause visual discomfort when the objects with large disparities are present in the scene. With this technique, the improvement of visual comfort has been reported by blurring far background and foregrounds in the scene. However, this technique has a drawback of degrading overall image quality. To lesson visual discomfort caused by large disparities while maintaining high-perceived image quality, we use a novel disparity-based asymmetrical filtering technique. Asymmetrical filtering, which refers to the filtering applied to the image of one eye only, has been showen to maintain the sharpness of a stereoscopic image, provided that the amount of filtering is low. Disparity-based asymmetrical filtering usese the disparity information in a stereoscopic image for controlling the severity of blurring. We investigated the effects of this technique on stereoscopic video by measuring visual comfort and apparent sharpness. Our results indicate that disparity-based asymmetrical filtering does not always improve visual comfort but it maintains image quality.

Visual comfort and apparent depth in 3D systems: effects of camera convergence distance

We investigated the effect of convergence of stereoscopic cameras on visual comfort and apparent depth. In Experiment 1, viewers rated comfort and depth of three stereoscopic sequences acquired with convergence distance set at 60, 120, 180, 240 cm, or infinity (i.e., parallel). Moderately converged conditions were rated either as comfortable (i.e., 240 cm) or more comfortable (i.e., 120 and 180 cm) than the parallel condition. The 60 cm condition was rated the least comfortable. Camera convergence had no effects on ratings of apparent depth. In Experiment 2, we used computer-generated stereoscopic still images to investigate the effects of convergence in the absence of lens distortions. Results matched those obtained in Experiment 1. In Experiment 3, we artificially introduced keystone distortions in stereoscopic still images. We found that increasing the amount of keystone distortion caused only a minimal decrease in visual comfort and apparent depth.

Stereo image quality: effects of spatio-temporal resolution

We explored the response of the human visual system to mixed-resolution stereo video sequences in which one eye view was spatially or temporally low-pass filtered. It was expected that perceived quality, stereo depth, and perceived sharpness of sequences would be relatively unaffected by low-pass filtering, compared to the case where both eyes viewed a filtered image. Subjects viewed two 10-second stereo video-sequences, in which the right-eye frames were filtered vertically and horizontally (H) at 1/2H, 1/2V, 1/4H, 1/4V, 1/2H1/2V, 1/2H1/2V, 1/4H1/2V, and 1/4H1/4V resolution. Temporal filtering was implemented for a subset of these conditions at 1/2 temporal resolution, or with dropped-and-repeated frames. Subjects rates the overall quality, sharpness, and overall sensation of depth. It was found that spatial filtering produced acceptable results: the overall sensation of depth was unaffected by low-pass filtering, while ratings of quality and of sharpness were biased towards the eye with the greater spatial resolution. By comparison, temporal filtering produced unacceptable result: Field averaging and dropped-and-repeated frame conditions yielded images with poor quality and sharpness, even though perceived depth was relatively unaffected. We conclude that spatial filtering of one channel of a stereo video-stereo may be an effective means of reducing transmission bandwidth.