Anne-Marie Brouwer

A Tactile P300 Brain-Computer Interface

In this study, we investigated a Brain-Computer Interface (BCI) based on EEG responses to vibro-tactile stimuli around the waist. P300 BCIs based on tactile stimuli have the advantage of not taxing the visual or auditory system and of being potentially unnoticeable to other people. A tactile BCI could be especially suitable for patients whose vision or eye movements are impaired. In Experiment 1, we investigated its feasibility and the effect of the number of equally spaced tactors. Whereas a large number of tactors is expected to enhance the P300 amplitude since the target will be less frequent, it could also negatively affect the P300 since it will be difficult to identify the target when tactor density increases. Participants were asked to attend to the vibrations of a target tactor, embedded within a stream of distracters. The number of tactors was two, four or six. We demonstrated the feasibility of a tactile P300 BCI. We did not find a difference in SWLDA classification performance between the different numbers of tactors. In a second set of experiments we reduced the stimulus onset asynchrony (SOA) by shortening the on- and/or off-time of the tactors. The SOA for an optimum performance as measured in our experiments turned out to be close to conventional SOAs of visual P300 BCIs.

Estimating workload using EEG spectral power and ERPs in the n-back task

Previous studies indicate that both electroencephalogram (EEG) spectral power (in particular the alpha and theta band) and event-related potentials (ERPs) (in particular the P300) can be used as a measure of mental work or memory load. We compare their ability to estimate workload level in a well-controlled task. In addition, we combine both types of measures in a single classification model to examine whether this results in higher classification accuracy than either one alone. Participants watched a sequence of visually presented letters and indicated whether or not the current letter was the same as the one (n instances) before. Workload was varied by varying n. We developed different classification models using ERP features, frequency power features or a combination (fusion). Training and testing of the models simulated an online workload estimation situation. All our ERP, power and fusion models provide classification accuracies between 80% and 90% when distinguishing between the highest and the lowest workload condition after 2 min. For 32 out of 35 participants, classification was significantly higher than chance level after 2.5 s (or one letter) as estimated by the fusion model. Differences between the models are rather small, though the fusion model performs better than the other models when only short data segments are available for estimating workload.

Perception of acceleration with short presentation times: Can acceleration be used in interception?

To investigate whether visual judgments of acceleration could be used for intercepting moving targets, we determined how well subjects can detect acceleration when the presentation time is short. In a differential judgment task, two dots were presented successively. One dot accelerated and the other decelerated. Subjects had to indicate which of the two accelerated. In an absolute judgment task, subjects had to adjust the motion of a dot so that it appeared to move at a constant velocity. The results for the two tasks were similar. For most subjects, we could determine a detection threshold even when the presentation time was only 300 msec. However, an analysis of these thresholds suggests that subjects did not detect the acceleration itself but that they detected that a target had accelerated on the basis of the change in velocity between the beginning and the end of the presentation. A change of about 25% was needed to detect acceleration with reasonable confidence. Perhaps the simplest use of acceleration for interception consists of distinguishing between acceleration and deceleration of the optic projection of an approaching ball to determine whether one has to run backward or forward to catch it. We examined the results of a real ball-catching task (Oudejans, Michaels, & Bakker, 1997) and found that subjects reacted before acceleration could have been detected. We conclude that acceleration is not used in this simple manner to intercept moving targets.

Hitting moving targets: a dissociation between the use of the target's speed and direction of motion.

Previous work has indicated that people do not use their judgment of a targets speed to determine where to hit it. Instead, they use their judgment of the targets changing position and an expected speed (based on the speed of previous targets). In the present study we investigate whether people also ignore the targets apparent direction of motion, and use the targets changing position and an expected direction of motion instead. Subjects hit targets that moved in slightly different directions across a screen. Sometimes the targets disappeared after 150xa0ms, long before the subjects could reach the screen. This prevented subjects from using the targets changing position to adjust their movements, making it possible to evaluate whether subjects were relying on the perceived or an expected (average) direction to guide their movements. The background moved perpendicular to the average direction of motion in some trials. This influences the targets perceived direction of motion while leaving its perceived position unaffected. When the background was stationary, subjects hit disappearing targets along their trajectory, just as they hit ones that remained visible. Moving the background affected the direction in which subjects started to move their hand, in accordance with the illusory change in direction of target motion. If the target disappeared, this resulted in a hit that was systematically off the targets trajectory. If the target remained visible, subjects corrected their initial error. Presumably they did so on the basis of information about the targets changing position, because if the target disappeared they did not correct the error. We conclude that people do use the targets perceived direction of motion to determine where to hit it. Thus the perceived direction of motion is treated differently than the perceived speed. This suggests that the motion of an object is not broken down into speed components in different directions, but that speed and direction are perceived and used separately.

Hitting moving objects: The dependency of hand velocity on the speed of the target.

Abstract. In previous studies, subjects did not hit slowly moving objects as quickly as fast ones, despite being instructed to hit them all as quickly as possible. In those studies the targets moved at constant but unpredictable velocities, and it has been suggested that subjects were unable to adjust the hands path to suit the velocity of the target. To compensate for this, they adjusted the speed of their hand to that of the target (speed coupling). According to this hypothesis, speed coupling is necessary only when subjects are unable to correctly predict the next target velocity and only if they have to be accurate. We show here that decreasing the uncertainty about the up-coming targets velocity or enlarging the tip of the hitting weapon does not make speed coupling disappear. Moreover, there is a negative correlation between hand velocity and strength of speed coupling, whereas the hypothesis predicts a positive correlation. The hypothesis is therefore rejected. We propose that speed coupling is a result of different speed-accuracy tradeoffs applying to different target velocities.

The role of memory in visually guided reaching

People can be shown to use memorized location information to move their hand to a target location if no visual information is available. However, for several reasons, memorized information may be imprecise and inaccurate. Here, we study whether and to what extent humans use the remembered location of an object to plan reaching movements when the target is visible. Subjects sequentially picked up and moved two different virtual, magnetic target objects from a target region into a virtual trash bin with their index fingers. In one third of the trials, we perturbed the position of the second target by 1 cm while the finger was transporting the first target to the trash. Subjects never noticed this. Although the second target was visible in the periphery, subjects movements were biased to its initial (remembered) position. The first part of subjects movements was predictable from a weighted sum of the visible and remembered target positions. For high contrast targets, subjects initially weigh visual and remembered information about target position in an average ratio of 0.67 to 0.33. Over the course of the movement, weight given to memory decreased. Diminishing the contrast of the targets substantially increased the weight that subjects gave to the remembered location. Thus, even when peripheral visual information is available, humans use the remembered location of an object to plan goal-directed movements. In contrast to previous suggestions in the literature, our results indicate that absolute location is remembered quite well.

Hitting moving objects: is target speed used in guiding the hand?

We investigated what information subjects use when trying to hit moving targets. In particular, whether only visual information about the target’s position is used to guide the hand to the place of interception or also information about its speed. Subjects hit targets that moved at different constant speeds and disappeared from view after varying amounts of time. This prevented the subjects from updating position information during the time that the target was invisible. Subjects hit further ahead of the disappearing point when the target moved faster, but not as much as they should have on the basis of the target’s speed. This could be because more time is needed to perceive and use the correct speed than was available before the target disappeared. It could also be due to a speed-related misperception of the target’s final position. The results of a second experiment were more consistent with the latter hypothesis. In a third experiment we moved the background to manipulate the perceived speed. This did not affect the hitting positions. We conclude that subjects respond only to the changing target position. Target speed influences the direction in which the hand moves indirectly, possibly via a speed-related misperception of position.

Differences in fixations between grasping and viewing objects

Where exactly do people look when they grasp an object? An object is usually contacted at two locations, whereas the gaze can only be at one location at the time. We investigated participants fixation locations when they grasp objects with the contact positions of both index finger and thumb being visible and compared these to fixation locations when they only viewed the objects. Participants grasped with the index finger at the top and the thumb at the bottom of a flat shape. The main difference between grasping and viewing was that after a saccade roughly directed to the objects center of gravity, participants saccaded more upward and more into the direction of a region that was difficult to contact during grasping. A control experiment indicated that it was not the upper part of the shape that attracted fixation, while the results were consistent with an attraction by the index finger. Participants did not try to fixate both contact locations. Fixations were closer to the objects center of gravity in the viewing than in the grasping task. In conclusion, participants adapt their eye movements to the need of the task, such as acquiring information about regions with high required contact precision in grasping, even with small (graspable) objects. We suggest that in grasping, the main function of fixations is to acquire visual feedback of the approaching digits.

Hitting moving targets: effects of target speed and dimensions on movement time.

To hit moving targets, one not only has to arrive at the right place but also at the right time. Moving quickly reduces spatial precision but increases temporal precision. This may explain why people usually move more quickly toward fast targets than toward slow ones, because arriving at the right time is more important when hitting fast targets. The temporal accuracy required depends not only on the target’s speed but also on its length in the direction of motion; it decreases with increasing length. Here we investigate the effects of variations in the target’s speed and dimensions on the subject’s movement time. We asked subjects to hit targets that moved from left to right as quickly as possible with their index finger. The targets varied in length in the direction of motion (width: affecting both spatial and temporal demands), in length in the orthogonal direction (height: affecting spatial demand), and in speed (affecting temporal demand). Targets were presented in random order during one session and in blocks of trials with identical targets during another session. In the latter session subjects could optimize their strategy for each target separately. In the random condition subjects hit fast targets more quickly than slow ones. Their movement time was also affected by the target’s size (the spatial demand), but not by the direction of the elongation. For the blocked condition, subjects did consider the direction of the elongation. We conclude that people do not consider an object’s orientation to estimate the temporal demands of an interception task, but that they use the object’s size and speed, and their experience from previous trials.

Humans use visual and remembered information about object location to plan pointing movements

We investigated whether humans use a targets remembered location to plan reaching movements to targets according to the relative reliabilities of visual and remembered information. Using their index finger, subjects moved a virtual object from one side of a table to the other, and then went back to a target. In some trials, the target shifted unnoticed while the finger made the first movement. We regressed subjects movement trajectories against the initial and shifted target locations to infer the weights that subjects gave to remembered and visual locations. We measured the reliability of vision and memory by adding conditions in which the target only appeared after subjects made the first movement (vision only) and in which the target was initially present but disappeared during the first movement (memory only). When both visual and remembered information were available, movement trajectories were biased to the remembered target location. The different weights that subjects gave to memory and visual information on average matched the weights predicted by the variance associated with the use of vision and memory alone. This suggests that humans integrate remembered information about object locations with peripheral visual information by taking into account the relative reliability of the two sources of information.