Heather L. Jenkin

Simulating Self-Motion I: Cues for the Perception of Motion

When people move there are many visual and non-visual cues that can inform them about their movement Simulating self-motion in a virtual reality environment thus needs to take these non-visual cues into account in addition to the normal high-quality visual display. Here we examine the contribution of visual and non-visual cues to our perception of self-motion. The perceived distance of self-motion can be estimated from the visual flow field, physical forces or the act of moving. On its own, passive visual motion is a very effective cue to self-motion, and evokes a perception of self-motion that is related to the actual motion in a way that varies with acceleration Passive physical motion turns out to be a particularly potent self-motion cue: not only does it evoke an exaggerated sensation of motion, but it also tends to dominate other cues.

Shape-from-shading depends on visual, gravitational, and body-orientation cues

The perception of shading-defined form results from an interaction between shading cues and the frames of reference within which those cues are interpreted. In the absence of a clear source of illumination, the definition of ‘up’ becomes critical to deducing the perceived shape from a particular pattern of shading. In our experiments, twelve subjects adjusted the orientation of a planar disc painted with a linear luminance gradient from one side to the other, until the disc appeared maximally convex—that is, until the luminance gradient induced the maximum perception of a three-dimensional shape. The vision, gravity, and body-orientation cues were altered relative to each other. Visual cues were manipulated by the York Tilted Room facility, and body cues were altered by simply lying on one side. The orientation of the disc that appeared maximally convex varied in a systematic fashion with these manipulations. We present a model in which the direction of perceptual ‘up’ is determined from the sum of three weighted vectors corresponding to the vision, gravity, and body-orientation cues. The model predicts the perceived direction of ‘up’, contributes to our understanding of how shape-from-shading is deduced, and also predicts the confidence with which the ‘up’ direction is perceived.

Perceptual stability during head movement in virtual reality

Virtual reality displays introduce spatial distortions that are very hard to correct because of the difficulty of precisely modelling the camera from the nodal point of each eye. How significant are these distortions for spatial perception in virtual reality? In this study, we used a helmet-mounted display and a mechanical head tracker to investigate the tolerance to errors between head motions and the resulting visual display. The relationship between the head movement and the associated updating of the visual display was adjusted by subjects until the image was judged as stable relative to the world. Both rotational and translational movements were tested, and the relationship between the movements and the direction of gravity was varied systematically. Typically, for the display to be judged as stable, subjects needed the visual world to be moved in the opposite direction to the head movement by an amount greater than the head movement itself, during both rotational and translational head movements, although a large range of movement was tolerated and judged as appearing stable. These results suggest that it not necessary to model the visual geometry accurately and suggest circumstances when tracker drift can be corrected by jumps in the display which will pass unnoticed by the user.

Perception of self-tilt in a true and illusory vertical plane

A tilted furnished room can induce strong visual reorientation illusions in stationary subjects. Supine subjects may perceive themselves upright when the room is tilted 90° so that the visual polarity axis is kept aligned with the subject. This ‘upright illusion’ was used to induce roll tilt in a truly horizontal, but perceptually vertical, plane. A semistatic tilt profile was applied, in which the tilt angle gradually changed from 0° to 90°, and vice versa. This method produced larger illusory self-tilt than usually found with static tilt of a visual scene. Ten subjects indicated self-tilt by setting a tactile rod to perceived vertical. Six of them experienced the upright illusion and indicated illusory self-tilt with an average gain of about 0.5. This value is smaller than with true self-tilt (0.8), but comparable to the gain of visually induced self-tilt in erect subjects. Apparently, the contribution of nonvisual cues to gravity was independent of the subjects orientation to gravity itself. It therefore seems that the gain of visually induced self-tilt is smaller because of lacking, rather than conflicting, nonvisual cues. A vector analysis is used to discuss the results in terms of relative sensory weightings.

The effect of altered gravity states on the perception of orientation.

We measured the effect of the orientation of the visual background on the perceptual upright (PU) under different levels of gravity. Brief periods of micro- and hypergravity conditions were created using two series of parabolic flights. Control measures were taken in the laboratory under normal gravity with subjects upright, right side down and supine. Participants viewed a polarized, natural scene presented at various orientations on a laptop viewed through a hood which occluded all other visual cues. Superimposed on the screen was a character the identity of which depended on its orientation. The orientations at which the character was maximally ambiguous were measured and the perceptual upright was defined as half way between these orientations. The visual background affected the orientation of the PU less when in microgravity than when upright in normal gravity and more when supine than when upright in normal gravity. A weighted vector sum model was used to quantify the relative influence of the orientations of gravity, vision and the body in determining the perceptual upright.

Enhancing visual cues to orientation: Suggestions for space travelers and the elderly

Establishing our orientation in the world is necessary for almost all aspects of perception and behavior. Gravity usually defines the critical reference direction. The direction of gravity is sensed by somatosensory detectors indicating pressure points and specialized organs in the vestibular system and viscera that indicate gravitys physical pull. However, gravitys direction can also be sensed visually since we see the effects of gravity on static and moving objects and also deduce its direction from the global structure of a scene indicated by features such as the sky and ground. When cues from either visual or physical sources are compromised or ambiguous, perceptual disorientation may result, often with a tendency to replace gravity with the bodys long axis as a reference. Orientation cues are compromised while floating in the weightlessness of space (which neutralizes vestibular and somatosensory cues) or while suspended at neutral buoyancy in the ocean (which neutralizes somatosensory cues) and the ability to sense orientation cues may also be compromised in the elderly or in clinical populations. In these situations, enhancing the visual cues to orientation may be beneficial. In this chapter, we review research using specially constructed virtual and real environments to quantify the contribution of various visual orientation cues. We demonstrate how visual cues can counteract disorientation by providing effective orientation information.

Perceptual Upright: The Relative Effectiveness of Dynamic and Static Images Under Different Gravity States

The perceived direction of up depends on both gravity and visual cues to orientation. Static visual cues to orientation have been shown to be less effective in influencing the perception of upright (PU) under microgravity conditions than they are on earth (Dyde et al., 2009). Here we introduce dynamic orientation cues into the visual background to ascertain whether they might increase the effectiveness of visual cues in defining the PU under different gravity conditions. Brief periods of microgravity and hypergravity were created using parabolic flight. Observers viewed a polarized, natural scene presented at various orientations on a laptop viewed through a hood which occluded all other visual cues. The visual background was either an animated video clip in which actors moved along the visual ground plane or an individual static frame taken from the same clip. We measured the perceptual upright using the oriented character recognition test (OCHART). Dynamic visual cues significantly enhance the effectiveness of vision in determining the perceptual upright under normal gravity conditions. Strong trends were found for dynamic visual cues to produce an increase in the visual effect under both microgravity and hypergravity conditions.

Asymmetrical representation of body orientation

The perceived orientation of objects, gravity, and the body are biased to the left. Whether this leftward bias is attributable to biases in sensing or processing vestibular, visual, and body sense cues has never been assessed directly. The orientation in which characters are most easily recognized--the perceived upright (PU)--can be well predicted from a weighted vector sum of these sensory cues. A simple form of this model assumes that the directions of the contributing inputs are coded accurately and as a consequence participants tilted left- or right-side-down relative to gravity should exhibit mirror symmetric patterns of responses. If a left/right asymmetry were present then varying these sensory cues could be used to assess in which sensory modality or modalities a PU bias may have arisen. Participants completed the Oriented Character Recognition Test (OCHART) while manipulating body posture and visual orientation cues relative to gravity. The response patterns showed systematic differences depending on which side they were tilted. An asymmetry of the PU was found to be best modeled by adding a leftward bias of 5.6° to the perceived orientation of the body relative to its actual orientation relative to the head. The asymmetry in the effect of body orientation is reminiscent of the body-defined left-leaning asymmetry in the perceived direction of light coming from above and reports that people tend to adopt a right-leaning posture.

The relative contributions of radial and laminar optic flow to the perception of linear self-motion

When illusory self-motion is induced in a stationary observer by optic flow, the perceived distance traveled is generally overestimated relative to the distance of a remembered target (Redlick, Harris, & Jenkin, 2001): subjects feel they have gone further than the simulated distance and indicate that they have arrived at a targets previously seen location too early. In this article we assess how the radial and laminar components of translational optic flow contribute to the perceived distance traveled. Subjects monocularly viewed a target presented in a virtual hallway wallpapered with stripes that periodically changed color to prevent tracking. The target was then extinguished and the visible area of the hallway shrunk to an oval region 40° (h) × 24° (v). Subjects either continued to look centrally or shifted their gaze eccentrically, thus varying the relative amounts of radial and laminar flow visible. They were then presented with visual motion compatible with moving down the hallway toward the target and pressed a button when they perceived that they had reached the targets remembered position. Data were modeled by the output of a leaky spatial integrator (Lappe, Jenkin, & Harris, 2007). The sensory gain varied systematically with viewing eccentricity while the leak constant was independent of viewing eccentricity. Results were modeled as the linear sum of separate mechanisms sensitive to radial and laminar optic flow. Results are compatible with independent channels for processing the radial and laminar flow components of optic flow that add linearly to produce large but predictable errors in perceived distance traveled.

The effect of long-term exposure to microgravity on the perception of upright

Going into space is a disorienting experience. Many studies have looked at sensory functioning in space but the multisensory basis of orientation has not been systematically investigated. Here, we assess how prolonged exposure to microgravity affects the relative weighting of visual, gravity, and idiotropic cues to perceived orientation. We separated visual, body, and gravity (when present) cues to perceived orientation before, during, and after long-term exposure to microgravity during the missions of seven astronauts on the International Space Station (mean duration 168 days) and measuring perceived vertical using the subjective visual vertical and the perceptual upright. The relative influence of each cue and the variance of their judgments were measured. Fourteen ground-based control participants performed comparable measurements over a similar period. The variance of astronauts’ subjective visual vertical judgments in the absence of visual cues was significantly larger immediately upon return to earth than before flight. Astronauts’ perceptual upright demonstrated a reduced reliance on visual cues upon arrival on orbit that re-appeared long after returning to earth. For earth-bound controls, the contributions of body, gravity, and vision remained constant throughout the year-long testing period. This is the first multisensory study of orientation behavior in space and the first demonstration of long-term perceptual changes that persist after returning to earth. Astronauts showed a plasticity in the weighting of perceptual cues to orientation that could form the basis for future countermeasures.Sensory perception: Knowing which way is ‘up’Prolonged exposure to microgravity has a long-term effect on the perception of upright. On earth we use visual, body, and gravity cues to help us determine the orientation of ourselves relative to the world which affects many perceptual tasks including reading, recognizing faces, and navigating. Laurence R. Harris and colleagues at York University assessed how seven astronauts who spent 168 days on average on the International Space Station perceived their orientation before, during and after flight. Although no changes were observed during their missions, astronauts’ judgements in the absence of visual cues were worse upon return to earth compared with ground-based controls. Harris and his team found that the effect persisted for up to four months after the astronauts returned to earth. These findings could help develop countermeasures to avoid perceptual mistakes during space travel, and contribute to facilitating safer, long-duration journeys without gravity.