Jonathan Matthis

Gaze and the Control of Foot Placement When Walking in Natural Terrain

SUMMARY Human locomotion through natural environments requires precise coordination between the biomechanics of the bipedal gait cycle and the eye movements that gather the information needed to guide foot placement. However, little is known about how the visual and locomotor systems work together to support movement through the world. We developed a system to simultaneously record gaze and full-body kinematics during locomotion over different outdoor terrains. We found that not only do walkers tune their gaze behavior to the specific information needed to traverse paths of varying complexity but that they do so while maintaining a constant temporal look-ahead window across all terrains. This strategy allows walkers to use gaze to tailor their energetically optimal preferred gait cycle to the upcoming path in order to balance between the drive to move efficiently and the need to place the feet in stable locations. Eye movements and locomotion are intimately linked in a way that reflects the integration of energetic costs, environmental uncertainty, and momentary informational demands of the locomotor task. Thus, the relationship between gaze and gait reveals the structure of the sensorimotor decisions that support successful performance in the face of the varying demands of the natural world.

The critical phase for visual control of human walking over complex terrain

Significance The physical dynamics of the body are central to the generation and maintenance of the human gait cycle. The ability to exploit the force of gravity and bodily inertia increases the energetic efficiency of locomotion by minimizing the need for internally generated muscular forces and simplifies control by obviating the need to actively guide each body segment. Here we explore how these principles generalize to situations in which foot placement is constrained, as when walking over a rocky trail. Walkers can exploit external forces to efficiently traverse extended stretches of complex terrain provided that visual information about the upcoming ground surface is available during a particular (critical) phase of the gait cycle between midstance of the preceding step and toe-off. To walk efficiently over complex terrain, humans must use vision to tailor their gait to the upcoming ground surface without interfering with the exploitation of passive mechanical forces. We propose that walkers use visual information to initialize the mechanical state of the body before the beginning of each step so the resulting ballistic trajectory of the walker’s center-of-mass will facilitate stepping on target footholds. Using a precision stepping task and synchronizing target visibility to the gait cycle, we empirically validated two predictions derived from this strategy: (1) Walkers must have information about upcoming footholds during the second half of the preceding step, and (2) foot placement is guided by information about the position of the target foothold relative to the preceding base of support. We conclude that active and passive modes of control work synergistically to allow walkers to negotiate complex terrain with efficiency, stability, and precision.

Control of gaze in natural environments: effects of rewards and costs, uncertainty and memory in target selection

The development of better eye and body tracking systems, and more flexible virtual environments have allowed more systematic exploration of natural vision and contributed a number of insights. In natural visually guided behaviour, humans make continuous sequences of sensory-motor decisions to satisfy current goals, and the role of vision is to provide the relevant information in order to achieve those goals. This paper reviews the factors that control gaze in natural visually guided actions such as locomotion, including the rewards and costs associated with the immediate behavioural goals, uncertainty about the state of the world and prior knowledge of the environment. These general features of human gaze control may inform the development of artificial systems.

Eye, head, and foot tracking during locomation over real-world complex terrain

Successful locomotion over complex terrain such as a rocky trail requires walkers to place each step on a safe foothold with high spatiotemporal precision while also taking the future terrain into account. We have previously shown that humans guide foot placement using a visual control strategy that exploits the physical dynamics of the bipedal gait cycle (e.g. Matthis, Barton & Fajen, 2014). However, this work was limited to locomotion over virtual terrain projected onto flat surfaces within the laboratory. In the current project, we develop a research protocol to allow accurate and integrated tracking of the eye, head, and feet during locomotion over real-world complex and rocky terrain, a protocol that has been previously limited by numerous technical challenges. This protocol allows us to test predictions derived from the results of previous lab-based studies on the visual control of foot placement in real-world terrains, and opens new opportunities to study the visual control of locomotion outside a laboratory setting. Our protocol utilizes a Positive Science mobile eye tracker with a GoPro scene camera and inertial measurement units (IMUs) attached to the head, trunk, and feet. Each IMU integrates the output of temperature-calibrated, tri-axial accelerometers, gyroscopes, and magnetometers to determine the sensors orientation within a world-centered reference frame. The orientation quaternions produced by the head and trunk-mounted IMUs were used to rotate the gaze vectors recorded by the eye tracker in order to specify gaze fixations within a body-centered reference frame and relate gaze location to foot placement. In a pilot study, subjects walked across different types of real-world terrain while performing different distraction tasks. Preliminary results reveal that foot placement in complex terrain is highly demanding of attentional resources, and that the control strategies used to guide foot placement vary dramatically according to the difficulty of the terrain being traversed. Meeting abstract presented at VSS 2015.

Towards understanding visually guided locomotion over complex and rough terrain: A phase-space planning method

As legged robots maneuver over increasingly complex and rough terrains, designing motion planners with the capability of predicting future footsteps becomes imperative. In turn, these planners provide a valuable tool for understanding the fundamental principles underlying human locomotion [2, 3]. In this study, we use our previously proposed phase-space planning framework [1] to analyze human walking over complex terrain. In particular, we highlight (i) the center of mass (CoM) apex-state-based feature of the phase-space planning, and (ii) the role of vision in CoM apex state selection during human walking over complex terrain [2].

Stereo camera reconstruction of rough terrain

This video shows a short clip of a 3D reconstruction of rough terrain by a stereo RGB camera setup.

Novel apparatus for investigation of eye movements when walking in the presence of 3D projected obstacles

The human gait cycle is incredibly efficient and stable largely because of the use of advance visual information to make intelligent selections of heading direction, foot placement, gait dynamics, and posture when faced with terrain complexity [Patla and Vickers 1997; Patla and Vickers 2003; Matthis and Fajen 2013; Matthis and Hayhoe 2015]. This is behaviorally demonstrated by a coupling between saccades and foot placement.

Visual information for the control of walking over complex terrain.

Which regions of the ground surface do humans need to see to control walking over complex terrain? Previously, we offered an answer to this question rooted in the biomechanics of walking: To efficiently exploit their inverted-pendulum-like structure, walkers should use information about potential target footholds for an upcoming step during the last part of the preceding step. That is, the last part of each step is the critical phase for the visual control of the upcoming step. The aim of the present study was to determine the nature of the visual information used during this critical control phase. To efficiently exploit their biomechanical structure, walkers must initialize each step with a pushoff force from the trailing foot that is properly tailored to the position of the next target relative to the previous target. This leads to the hypothesis that walkers rely on information about the relative position of pairs of consecutive targets. To test this hypothesis, we instructed subjects to walk along a path of irregularly spaced target footholds (small circular patches of light projected onto the floor) while their movements were tracked by a motion capture system. On some trials, the visibility of a subset of targets was manipulated such that they were only visible for a brief period. The duration of the period of visibility varied such that consecutive targets were simultaneously visible in some conditions (leaving relative position information intact) but not others. We found no significant differences in stepping accuracy between conditions in which relative position information was available and a control condition in which all targets were always visible. However, stepping accuracy degraded in conditions in which relative position information was unavailable. We conclude that walkers rely on relative position information and that such information facilitates energetically efficient walking over complex terrain. Meeting abstract presented at VSS 2015.