Don E. Pearson

Using Behavioral Realism to Estimate Presence: A Study of the Utility of Postural Responses to Motion Stimuli

We recently reported that direct subjective ratings of the sense of presence are potentially unstable and can be biased by previous judgments of the same stimuli (Freeman et al., 1999). Objective measures of the behavioral realism elicited by a display offer an alternative to subjective ratings. Behavioral measures and presence are linked by the premise that, when observers experience a mediated environment (VE or broadcast) that makes them feel present, they will respond to stimuli within the environment as they would to stimuli in the real world. The experiment presented here measured postural responses to a video sequence filmed from the hood of a car traversing a rally track, using stereoscopic and monoscopic presentation. Results demonstrated a positive effect of stereoscopic presentation on the magnitude of postural responses elicited. Posttest subjective ratings of presence, vection, and involvement were also higher for stereoscopically presented stimuli. The postural and subjective measures were not significantly correlated, indicating that nonproprioceptive postural responses are unlikely to provide accurate estimates of presence. Such postural responses may prove useful for the evaluation of displays for specific applications and in the corroboration of group subjective ratings of presence, but cannot be taken in place of subjective ratings.

Effects of Sensory Information and Prior Experience on Direct Subjective Ratings of Presence

We report three experiments using a new form of direct subjective presence evaluation that was developed from the method of continuous assessment used to assess television picture quality. Observers were required to provide a continuous rating of their sense of presence using a handheld slider. The first experiment investigated the effects of manipulating stereoscopic and motion parallax cues within video sequences presented on a 20 in. stereoscopic CRT display. The results showed that the presentation of both stereoscopic and motion parallax cues was associated with higher presence ratings. One possible interpretation of Experiment 1 is that CRT displays that contain the spatial cues of stereoscopic disparity and motion parallax are more interesting or engaging. To test this, observers in Experiment 2 rated the same stimuli first for interest and then for presence. The results showed that variations in interest did not predict the presence ratings obtained in Experiment 1. However, the subsequent ratings of presence differed significantly from those obtained in Experiment 1, suggesting that prior experience with interest ratings affected subsequent judgments of presence. To test this, Experiment 3 investigated the effects of prior experience on presence ratings. Three groups of observers rated a training sequence for interest, presence, and 3-Dness before rating the same stimuli as used for Experiments 1 and 2 for presence. The results demonstrated that prior ratings sensitize observers to different features of a display resulting in different presence ratings. The implications of these results for presence evaluation are discussed, and a combination of more-refined subjective measures and a battery of objective measures is recommended.

Viewer response to time-varying video quality

Over the past several years we have investigated viewer response to temporal fluctuations in the quality of digital television pictures, which occur when video is coded into relatively low bit rates. Three phenomena of interest have been identified: (1) a forgiveness effect, (2) a recency effect, and (3) a negative-peak (duration-neglect); these are described and discussed in the paper. In collaboration with our partners in European projects MOSAIC and TAPESTRIES, we have developed a three-stage method of measuring time-variant quality, which has been accepted by the ITU-R. The first stage is a Single Stimulus Continuous Quality Evaluation (SSCQE) of instantaneous quality; the second a calibration stage to link SSCQE with conventional DSCQS, and the third stage a numerical procedure for relating continuous and overall quality. Some of the factors we have identified as being important in producing good overall quality judgements have relevance to the design of optimal coding strategies for digital television.

Visual Communication Systems for the Deaf

The paper is in two parts. The first (Section II) describes six years operating experience with an in-house switched videotelephone system installed in the headquarters of the Royal National Institute for the Deaf, London. The system, which has recently been extended to 10 terminals, has confirmed the usefulness and practicality of switched television for both deaf-to-hearing and deaf-to-deaf communication. Proper attention is required, however, to terminal design and maintenance procedures. In the second part (Sections III and IV) consideration is given to extending the system to permit communication over the public switched telephone network. This demands considerable data compression of the images. Experiments are reported using six deaf subjects in which the tolerance of signing, lip-reading, and fingerspelling to temporal, spatial, and contrast coding distortions was investigated. The results suggest that just-comfortable communication can be achieved at around 100 kbits/s and just-possible communication at around 5 kbits/s, with further reductions appearing feasible.

Texture mapping in model-based image coding

Abstract An analysis is given of texture storage and reprojection as used in model-based image coding. It is shown that changes in object surface orientation give rise to combined amplitude and frequency modulation of the surface texture when projected onto the image plane. When stored texture is reprojected onto a model, inaccuracies in the model shape cause texture displacements under rotational motion. The use of these results to improve model-based prediction in practical systems is discussed.

A calibration method for continuous video quality (SSCQE) measurements

It is shown that single stimulus continuous quality evaluation (SSCQE) measurements can be calibrated against a standard reference, even though they relate to long video sequences of up to 25 min duration. This can be accomplished by projecting chosen percentiles in the histogram of continuous quality judgements onto the SSCQE graph, so identifying appropriate representative sections for use in DSCQS tests. Three different algorithms were tested for detailed selection of the 10 s sections, with a modal quality criterion giving the best results. Calibration using both linear and non-linear mapping between SSCQE and MDS scores has been satisfactorily accomplished.

Variability of performance in video coding

Modern video compression techniques exhibit variability of performance as a function of time. Studies are reported of viewer reactions to this variability, which indicates a sensitivity to particular features. Some interesting conclusions emerge for future work in video coding.

Sensitivity of model-based coded video to packet losses

Previous model-based video coding studies have focused on the modelling techniques themselves and have not considered the transportation of an implementable coder over lossy networks. In this article a hybrid switched 3D model-based/H.261 video coder designed for one-to-many distance learning applications over the Internet multicast backbone (MBONE) is employed to study the susceptibility to data loss of model based packet video. Data is considered as; texture, motion and shape, with further sub-division to periodic and aperiodic types. The absence of the various data streams in isolation is investigated to reveal the types of degradation possible and then statistical loss is applied to all streams simultaneously to observe the combined effects using a simple loss model. The ability of the various streams to recover from these errors is investigated and an order of priority for the data types is then identified.

Low Data-Rate Coding Using Image Primitives

We report on some initial attempts to reconstruct a grey-level image starting trom a cartoon or sketch, which is treated as an image primitive. The aim is to code grey-Level images efficiently by using object-related information. It the cartoon is derived t valley detection from the image of a smooth object, samples of the original image taken at the cartoon lines and at a few selected intermediate points contain implicit inrormation about the geometry of the object. Some initial experimental results with (a) simple black-level fill-in, and (b) first-order interpolation have been obtained.

PETAL: A parallel processor for real-time primitive extraction

A real-time parallel processor named PETAL is described which has been developed to extract cartoon primitives from grey-level television images. It is based on a cascaded look-up table architecture and is controlled by a 68000 microcomputer. It can process 256x256 images at 50 frames/s.