Wa James Tam

Stereo image quality: effects of mixed spatio-temporal resolution

We explored the response of the human visual system to mixed-resolution stereo video-sequences, in which one eye view was spatially or temporally low-pass filtered. It was expected that the perceived quality, depth, and sharpness would be relatively unaffected by low-pass filtering, compared to the case where both eyes viewed a filtered image. Subjects viewed two 10-second stereo video-sequences, in which the right-eye frames were filtered vertically (V) and horizontally (H) at 1/2 H, 1/2 V, 1/4 H, 1/4 V, 1/2 H 1/2 V, 1/2 H 1/4 V, 1/4 H 1/2 V, and 1/4 H 1/4 V resolution. Temporal filtering was implemented for a subset of these conditions at 1/2 temporal resolution, or with drop-and-repeat frames. Subjects rated the overall quality, sharpness, and overall sensation of depth. It was found that spatial filtering produced acceptable results: the overall sensation of depth was unaffected by low-pass filtering, while ratings of quality and of sharpness were strongly weighted towards the eye with the greater spatial resolution. By comparison, temporal filtering produced unacceptable results: field averaging and drop-and-repeat frame conditions yielded images with poor quality and sharpness, even though perceived depth was relatively unaffected. We conclude that spatial filtering of one channel of a stereo video-sequence may be an effective means of reducing the transmission bandwidth.

Psychovisual aspects of viewing stereoscopic video sequences

In comparison to conventional displays, 3D stereoscopic displays convey additional information about the 3D structure of a scene by providing information that can be used to extract depth. In the present study we evaluated the psychovisual impact of stereoscopic images on viewers. Thirty-three non-expert viewers rated sensation of depth, perceived sharpness, subjective image quality, and relative preference for stereoscopic over non-stereoscopic images. Rating methods were based on procedures described in ITU- Rec. 500. Viewers also rated sequences in which the left- and right-eye images were processed independently, using a generic MPEG-2 codec, at bit-rates of 6, 3, and 1 Mbits/s. The main finding was that viewers preferred the stereoscopic version over the non-stereoscopic version of the sequences, provided that the sequence did not contain noticeable stereo artifacts, such as exaggerated disparity. Perceived depth was rated greater for stereoscopic than for non-stereoscopic sequences, and perceived sharpness of stereoscopic sequences was rated the same or lower compared to non-stereoscopic sequences. Subjective image quality was influenced primarily by apparent sharpness of the video sequences, and less so by perceived depth.

Viewing behavior: ocular and attentional disengagement.

In five experiments, we examined the role of the ocular and attentional systems in determining saccadic latencies. Prior to making a saccade to a target stimulus, subjects were required to direct their attention to a foveal stimulus or to an eccentric stimulus. Either stimulus could be extinguished before the onset of the target. Saccadic latencies were shortest when the foveal stimulus was extinguished, regardless of whether it was attended or not. Control experiments showed that subjects were able to attend properly and that warning, arising from turning off a stimulus before target onset, could not completely account for the results. The results were discussed in terms of ocular disengagement, attentional disengagement, and joint ocular-attentional disengagement. It was concluded that an explanation emphasizing ocular disengagement provided the best account of the data.

Human perception of mismatched stereoscopic 3D inputs

The bandwidth required to transmit stereoscopic video images is nominally twice that required for standard, monoscopic images. One method of reducing the required bandwidth is to code the two video streams asymmetrically. We assessed the perceptual impact of this bandwidth-reduction technique for low-pass filtering, DCT-based quantization, and a combination of filtering and quantization. It was found that the binocular percept depended on the type of degradation: for low-pass filtering, the binocular percept was dominated by the high-quality image, whereas for quantization it corresponded to the average of the inputs to the two eyes. The results indicated that asymmetrical coding is a promising technique for reducing storage and transmission bandwidth of stereoscopic sequences.

Stereoscopic video: asymmetrical coding with temporal interleaving

Asymmetrical coding has been shown to be a viable method for reducing the bandwidth required for stereoscopic video storage and transmission. In the basic version of asymmetric coding, high quality images are streamed to one eye, and lower quality images are streamed to the other eye. To remove this imbalance in image quality between the two eyes, we propose a modified version of asymmetrical coding where high-quality images are interleaved with reduced-quality images within each stream. The change between high-quality and reduced-quality images occurs in counter-phase for the two image streams, such that the levels of image quality are cross-switched between streams. Experimental evidence is provided to show that a cross-switch is best positioned at scene cuts where it is masked, otherwise it is visible as a jerky motion in the stereoscopic picture. We conclude that a modified version of asymmetric coding with cross-switches occurring at scene-cuts is a useful method for balancing the image quality between eyes without introducing artifacts, while maintaining the feature of bandwidth reduction for stereoscopic video storage and transmission.

Perceptual Basis of Stereoscopic Video

We reviewed studies of viewers reactions to stereoscopic image sequences. The dimensions considered were perceived image quality, sharpness, depth and naturalness. Stereoscopic displays produced a reliable and consistent increase in the perceived depth of image sequences. By comparison, improvements on other dimensions were not as robust. The key conclusion is that viewers responses to stereoscopic image sequences vary along a number of independent dimensions. Overall preference for stereoscopic images will occur only if the enhanced depth perceived in a stereoscopic image sequence is not accompanied by distortions created by excessive disparity, ghosting/crosstalk, or conflicts between monoscopic and stereoscopic depth information.

Bandwidth reduction for stereoscopic video signals

The bandwidth required to transmit stereoscopic video signals is nominally twice that required for standard, monoscopic images. An effective method of reducing the required bandwidth is to code the two video streams asymmetrically. We assessed the impact of this bandwidth- reduction technique on image quality and overall sensation of depth. Images from the right-eye stream were spatially filtered on image quality and overall sensation of depth. Images from the right-eye stream were stream were spatially filtered to half and quarter resolution. Subsequently, the images were processed using an MPEG-2 codec at bit-rates of 6, 2, and 1 Mbit/s. Subjects assessed image quality and depth using a double-stimulus, continuous-quality scale method. It was found that perceived depth was relatively robust to spatial filtering and bit-rate reduction. Image quality was affected more by bit-rate reduction than by spatial filtering and, at the lower bit rates, ratings were much higher for stereoscopic than for non-stereoscopic sequences. The results indicate that asymmetrical coding of stereoscopic sequences can be an effective means of reducing bandwidth for storage and transmission.

Stereo-anomalous vision in a sample of young adults

We estimated the proportion of people that have defective stereo vision and are unable to utilize stereo disparity information to perceive depth. Previous estimates have ranged anywhere from as high as 30% to as low as 6%. Our goal was to understand the basis for the wide range in these estimates. To do this, we administered two psychophysical tests to a sample of 100 young adults. Visual stimuli consisted of dynamic random-dot stereograms presented using a fast-decay, time-sequential display device. The stimuli covered a range of disparities between 0 and .38 degrees (both crossed and uncrossed). A forced-choice methodology was used to estimate whether subjects could perceive depth based on horizontal disparity. It was found that display duration was a key variable determining the number of viewers that were classified as stereo-anomalous. The relatively high incidence of stereo- anomalous viewers in previous research was explained by the short display durations (80 ms) used in those studies. With longer durations of about 1 sec, we found that only about 5% of viewers had defective stereo vision.

Stereo image quality: effects of spatio-temporal resolution

We explored the response of the human visual system to mixed-resolution stereo video sequences in which one eye view was spatially or temporally low-pass filtered. It was expected that perceived quality, stereo depth, and perceived sharpness of sequences would be relatively unaffected by low-pass filtering, compared to the case where both eyes viewed a filtered image. Subjects viewed two 10-second stereo video-sequences, in which the right-eye frames were filtered vertically and horizontally (H) at 1/2H, 1/2V, 1/4H, 1/4V, 1/2H1/2V, 1/2H1/2V, 1/4H1/2V, and 1/4H1/4V resolution. Temporal filtering was implemented for a subset of these conditions at 1/2 temporal resolution, or with dropped-and-repeated frames. Subjects rates the overall quality, sharpness, and overall sensation of depth. It was found that spatial filtering produced acceptable results: the overall sensation of depth was unaffected by low-pass filtering, while ratings of quality and of sharpness were biased towards the eye with the greater spatial resolution. By comparison, temporal filtering produced unacceptable result: Field averaging and dropped-and-repeated frame conditions yielded images with poor quality and sharpness, even though perceived depth was relatively unaffected. We conclude that spatial filtering of one channel of a stereo video-stereo may be an effective means of reducing transmission bandwidth.

Is monocular degradation visible in fused stereo images

For efficient transmission of stereoscopic images over bandwidth-limited channels, human factors specialists have recognized that savings can be achieved by degrading one monocular component of a stereo pair and maintaining the other at the desired quality. The desired quality can be preserved as long as binocular vision assigns greater weight to the non-degraded component. The present study sought to determine if such over-weighting occurred when the monocular degradation included blocking artifacts common to DCT-based compression at low bit-rates. Stereo images with asymmetric amounts of degradation in the left and right components were matched to symmetric images on a metric of blocking artifact visibility. Under weighting of the higher-quality component was indicated because matches did not require that the degree of improvement in one component be offset by equivalent degradation of the other. These results suggest that blocking artifacts should not be present if monocular degradation is to be a successful means of bandwidth savings for stereo image transmission. There was also evidence that the type of weighting can depend upon which eye is shown the higher-quality component, suggesting the ta monocular degradation should not be applied to only one eye.