Erich Schneider

Eyeseecam: an eye movement-driven head camera for the examination of natural visual exploration

The prototype of a gaze‐controlled, head‐mounted camera (EyeSeeCam) was developed that provides the functionality for fundamental studies on human gaze behavior even under dynamic conditions like locomotion. EyeSeeCam incorporates active visual exploration by saccades with image stabilization during head, object, and surround motion just as occurs in human ocular motor control. This prototype is a first attempt to combine free user mobility with image stabilization and unrestricted exploration of the visual surround in a man‐made technical vision system. The gaze‐driven camera is supplemented by an additional wide‐angle, head‐fixed scene camera. In this scene view, the focused gaze view is embedded with picture‐in‐picture functionality, which provides an approximation of the foveated retinal content. Such a combined video clip can be viewed more comfortably than the saccade‐pervaded image of the gaze camera alone. EyeSeeCam consists of a video‐oculography (VOG) device and a camera motion device. The benchmark for the evaluation of such a device is the vestibulo‐ocular reflex (VOR), which requires a latency on the order of 10 msec between head and eye (camera) movements for proper image stabilization. A new lightweight VOG was developed that is able to synchronously measure binocular eye positions at up to 600 Hz. The camera motion device consists of a parallel kinematics setup with a backlash‐free gimbal joint that is driven by piezo actuators with no reduction gears. As a result, the latency between the rotations of an artificial eye and the camera was 10 msec, which is VOR‐like.

Gaze allocation in natural stimuli: Comparing free exploration to head-fixed viewing conditions

“Natural” gaze is typically measured by tracking eye positions during scene presentation in laboratory settings. How informative are such investigations for real-world conditions? Using a mobile eyetracking setup (“EyeSeeCam”), we measure gaze during free exploration of various in- and outdoor environments, while simultaneously recording head-centred videos. Here, we replay these videos in a laboratory setup. Half of the laboratory observers view the movies continuously, half as sequences of static 1-second frames. We find a bias of eye position to the stimulus centre, which is strongest in the 1 s frame replay condition. As a consequence, interobserver consistency is highest in this condition, though not fully explained by spatial bias alone. This leaves room for image specific bottom-up models to predict gaze beyond generic biases. Indeed, the “saliency map” predicts eye position in all conditions, and best for continuous replay. Continuous replay predicts real-world gaze better than 1 s frame replay does. In conclusion, experiments and models benefit from preserving the spatial statistics and temporal continuity of natural stimuli to improve their validity for real-world gaze behaviour.

Visual attention of anaesthetists during simulated critical incidents

BACKGROUNDnSituation awareness (SA) is considered to be an important non-technical skill for delivering safe anaesthesia. The spatial distribution of visual attention (VA) is an underlying process for attaining adequate SA. In the present study, a novel technology was used to assess the distribution of VA in anaesthetists delivering anaesthesia. The impact of a critical incident on VA in relation to individual experience is analysed in a descriptive and exploratory manner.nnnMETHODSnFifteen anaesthetists induced general anaesthesia in a full-scale simulator while wearing a head-mounted eye-tracking camera system. After an uneventful session, workload was increased in a randomized order by simulation of a critical incident in the second or third session. Eye tracking was used for the assessment of individuals distribution of VA to monitors, patient, and environment. A post hoc video analysis revealed information about the spatial distribution of VA. Descriptive statistics and exploratory analysis were used.nnnRESULTSnTwenty per cent of VA was directed to the patient monitor (30% during critical incident scenarios, P=0.003). The more experienced anaesthetists (more than 2 yr of work experience) increased the amount of time dedicated to manual tasks from 21% to 25% during critical incidents, whereas the less experienced decreased from 20% to 14% (P=0.061).nnnCONCLUSIONSnDistribution of attention is different during anaesthesia induction with critical incidents compared with uneventful anaesthesia induction. Less experienced anaesthesia providers spend more time on monitoring tasks. Further investigation in confirmatory designs is needed.

Comparison of 10-mg doses of 4-aminopyridine and 3,4-diaminopyridine for the treatment of downbeat nystagmus.

Objective Animal experiments have demonstrated that aminopyridines increase Purkinje cell excitability, and in clinical studies, 4-aminopyridine (4-AP) and 3,4-diaminopyridine (3,4-DAP) improved downbeat nystagmus. In this double-blind, prospective, crossover study, the effects of equivalent doses of 4-AP and 3,4-DAP on the slow-phase velocity (SPV) of downbeat nystagmus were compared. Methods Eight patients with downbeat nystagmus due to different etiologies (cerebellar degeneration [n = 1], bilateral vestibulopathy [n = 1], bilateral vestibulopathy and cerebellar degeneration [n = 1], Arnold-Chiari I malformation and cerebellar ataxia [n = 1], cryptogenic cerebellar ataxia [n = 4]) were included. They were randomly assigned to receiving a single capsule of 10 mg of 3,4-DAP or 4-AP followed by 6 days with no medication. One week later, the treatment was switched, that is, 1 single capsule (10 mg) of the other agent. Recordings with 3-dimensional video-oculography were performed before and 45 and 90 minutes after drug administration. Results Both medications had a significant effect throughout time (pre vs post 45 vs post 90) (F(2,14) = 8.876; P < 0.01). Following the administration of 3,4-DAP, mean slow velocity decreased from −5.68°/s (pre) to −3.29°/s (post 45) to −2.96°/s (post 90) (pre vs post 45/post 90 P < 0.01). In 4-AP, the mean SPV decreased from −6.04°/s (pre) to −1.58°/s (post 45) to −1.21°/s (post 90) (pre vs post 45/post 90 P < 0.00001). Both after 45 and after 90, the mean SPVs were significantly lower for 4-AP than for 3,4-DAP (P < 0.05). None of the patients reported serious side effects. Conclusion Based on these results, 10-mg doses of 4-AP lead to a more pronounced decrease of the SPV of downbeat nystagmus than do equivalent doses of 3,4-DAP.

Head position during resting modifies spontaneous daytime decrease of downbeat nystagmus.

Background: The intensity of downbeat nystagmus (DBN) decreases during the daytime when the head is in upright position. Objective: This prospective study investigated whether resting in different head positions (upright, supine, prone) modulates the intensity of DBN after resting. Methods: Eye movements of 9 patients with DBN due to cerebellar (n = 2) or unknown etiology (n = 7) were recorded with video-oculography. Mean slow-phase velocities (SPV) of DBN were determined in the upright position before resting at 9 am and then after 2 hours (11 am) and after 4 hours (1 pm) of resting. Whole-body positions during resting were upright, supine, or prone. The effects of all 3 resting positions were assessed on 3 separate days in each patient. Results: Before resting (9 am), the average SPV ranged from 3.05 °/s to 3.6 °/s on the separate days of measurement. After resting in an upright position, the average SPV at 11 am and 1 pm was 0.65 °/sec, which was less (p < 0.05) than after resting in supine (2.1 °/sec) or prone (2.22 °/sec) positions. Conclusion: DBN measured during the daytime in an upright position becomes minimal after the patient has rested upright. The spontaneous decrease of DBN is less pronounced when patients lie down to rest. This indicates a modulation by otolithic input. We recommend that patients with DBN rest in an upright position during the daytime. Classification of evidence: This study provides Class II evidence that for patients with DBN 2 hours of rest in the upright position decreases nystagmus more than 2 hours of rest in the supine or prone positions (relative improvement 79% upright, 33% supine, and 38% prone: p < 0.05).

Low-latency combined eye and head tracking system for teleoperating a robotic head in real-time

We have developed a low-latency combined eye and head tracker suitable for teleoperating a remote robotic head in real-time. Eye and head movements of a human (wizard) are tracked and replicated by the robot with a latency of 16.5 ms. The tracking is achieved by three fully synchronized cameras attached to a head mount. One forward-looking, wide-angle camera is used to determine the wizards head pose with respect to the LEDs on the video monitor; the other two cameras are for binocular eye tracking. The whole system operates at a sample rate of 220 Hz, which allows the capture and reproduction of biological movements as precisely as possible while keeping the overall latency low. In future studies, this setup will be used as an experimental platform for Wizard-of-Oz evaluations of gaze-based human-robot interaction. In particular, the question will be addressed as to what extent aspects of human eye movements need to be implemented in a robot in order to guarantee a smooth interaction.

Experimental platform for Wizard-of-Oz evaluations of biomimetic active vision in robots

A novel paradigm for the evaluation of humanrobot interaction is proposed, with special focus on the importance of natural eye and head movements in nonverbal human-machine communication scenarios. We present an experimental platform that will enable Wizard-of-Oz experiments in which a human experimenter (wizard) teleoperates a robotic head and eyes with his own head and eyes. Since the robot is animated based on the nonverbal behaviors of the human experimenter, the whole range of human eye movements can be presented without having to implement a complete gaze behavior model first. The experimenter watches and reacts to the video stream of the participant who directly interacts with the robot. Results are presented that focus on those technical aspects of the experimental platform that enable real-time and human-like interaction capabilities. In particular, the tracking of ocular motor dynamics, its replication in a robotic active vision system, and the involved teleoperation delays are evaluated. This setup will help to answer the question of which aspects of human gaze and head movement behavior have to be implemented to achieve humanness in active vision systems of robots.

Manual motor control during “virtual” self-motion: Implications for VR rehabilitation

The level of immersion that is induced in an individual can be measured by subjective report, but VE immersion can also affect automatic sensorimotor processes which function below perceptual thresholds. Such sub-threshold effects on central nervous system processing are important to understand for the purposes of shaping VE rehabilitation. This study investigates the effect of dynamic immersive VE on self-motion perception and automatic upper extremity motor response. Subjects viewed either horizontal or vertical sinusoidal linear translation ±1m at 0.25 Hz via a head-mounted display (HMD) while sitting in a stationary motion apparatus. Subjects performed a perceptuomotor task of aligning a handheld object to perceived vertical using their unconstrained arm (i.e. free to move in 6 DOF). Two objects were tested, a joystick and a full glass of water, in counter-balanced order. Results show the majority of subjects perceive self-motion that spatially and temporally agrees with the visually depicted VE motion. This occurs despite the absence of sinusoidally varying changes to gravitoinertial forces, since subjects are not exposed to actual physical motion. Despite only being instructed to orient the handheld object, handheld object kinematics also show automatic motor responses involving object translation. These manual motor responses were dependent on the direction and phase of the visual motion depicted in the VE. Specifically, vertical visual motion induced vertical translation and pitch tilt of the handheld object, while horizontal visual motion induced horizontal translation and roll tilt of the object. Motor responses were significantly greater in subjects who reported compelling self-motion perception. These findings suggest that a representation of net gravitoinertial forces can be derived from the high-fidelity, pictorial and dynamic depth cues visually presented in a VE. Automatic upper extremity manual responses which are controlled by descending central systems and tracts dissociable from lower extremities can be affected by immersion in a VE much like automatic postural behavior has been shown to be. This new evidence supports current efforts to conduct upper extremity rehabilitation in the relative safe and controllable experimental environments that VE technology affords.