Stephanie Huette

Continuous dynamics of color categorization

In order to assess whether color categorization is sensitive to within-category differences in hue, we monitored mouse trajectories in a modified categorization task. Participants saw color swatches from a blue-green continuum and categorized them with a computer mouse by selecting one of two colored regions at the top of a monitor. An analysis of the mouse trajectories showed that the deviation toward the competing category was a function of hue: As hues approached the category boundary, they increasingly deviated to the competitor. This work presents evidence for parallel activation on the level of hue and category processing for color, as well as simultaneous activation of perceptually adjacent categories. Thus, a dynamic process sensitive to fine-grained within-category detail best characterizes color categorization.

Degree of certainty modulates anticipatory processes in real time.

In the present study, we investigated how degree of certainty modulates anticipatory processes using a modified spatial cuing task in which participants made an anticipatory hand movement with the computer mouse toward one of two probabilistic targets. A cue provided information of the location of the upcoming target with 100% validity (certain condition), 75% validity (semicertain condition) or gave no information of the location (uncertain condition). We found that the degree of certainty associated with the probabilistic precue on the upcoming target location affected the spatiotemporal characteristics of the anticipatory hand movements in a systematic way. In the case of semicertainty, we found evidence that the anticipatory processes were modulated in a way consistent with a model of graded probability matching biased toward certainty. In the case of uncertainty regarding two equally likely locations, we observed large between- and within-subject variability in the patterns of anticipatory hand movements, suggesting that individual differences in the strategies employed may become relevant when the likelihoods of response options are equal.

Eye movements during listening reveal spontaneous grammatical processing.

Recent research using eye-tracking typically relies on constrained visual contexts in particular goal-oriented contexts, viewing a small array of objects on a computer screen and performing some overt decision or identification. Eyetracking paradigms that use pictures as a measure of word or sentence comprehension are sometimes touted as ecologically invalid because pictures and explicit tasks are not always present during language comprehension. This study compared the comprehension of sentences with two different grammatical forms: the past progressive (e.g., was walking), which emphasizes the ongoing nature of actions, and the simple past (e.g., walked), which emphasizes the end-state of an action. The results showed that the distribution and timing of eye movements mirrors the underlying conceptual structure of this linguistic difference in the absence of any visual stimuli or task constraint: Fixations were shorter and saccades were more dispersed across the screen, as if thinking about more dynamic events when listening to the past progressive stories. Thus, eye movement data suggest that visual inputs or an explicit task are unnecessary to solicit analog representations of features such as movement, that could be a key perceptual component to grammatical comprehension.

Processing motion implied in language: eye-movement differences during aspect comprehension.

Previous research on language comprehension has used the eyes as a window into processing. However, these methods are entirely reliant upon using visual or orthographic stimuli that map onto the linguistic stimuli being used. The potential danger of this method is that the pictures used may not perfectly match the internal aspects of language processing. Thus, a method was developed in which participants listened to stories while wearing a head-mounted eyetracker. Preliminary results demonstrate that this method is uniquely suited to measure responses to stimuli in the absence of visual stimulation.

Negation without symbols: the importance of recurrence and context in linguistic negation

A simple recurrent network with a perceptual simulation layer was trained on a corpus of affirmative and negated sentences. Linguistic negation can be encoded by the network via the inclusion (or absence) of features and categories associated with the senses, in one step, without the need for an explicit logical operation or for treating the negating word any differently than any other words. Visualizing negation as a trajectory in perceptual simulation space is explored in detail, and the implications for artificial intelligence, embodied computational models, and more practical implications of everyday use of negations are discussed.

Drawing from Memory: Hand-Eye Coordination at Multiple Scales

Eyes move to gather visual information for the purpose of guiding behavior. This guidance takes the form of perceptual-motor interactions on short timescales for behaviors like locomotion and hand-eye coordination. More complex behaviors require perceptual-motor interactions on longer timescales mediated by memory, such as navigation, or designing and building artifacts. In the present study, the task of sketching images of natural scenes from memory was used to examine and compare perceptual-motor interactions on shorter and longer timescales. Eye and pen trajectories were found to be coordinated in time on shorter timescales during drawing, and also on longer timescales spanning study and drawing periods. The latter type of coordination was found by developing a purely spatial analysis that yielded measures of similarity between images, eye trajectories, and pen trajectories. These results challenge the notion that coordination only unfolds on short timescales. Rather, the task of drawing from memory evokes perceptual-motor encodings of visual images that preserve coarse-grained spatial information over relatively long timescales as well.

Gesture variants and cognitive constraints for interactive virtual reality training systems

Two studies investigated the nature of environmental context on various parameters of pointing. The results revealed the need for extreme temporal precision and the need for efficient algorithms to parse out different styles of pointing. Most variability in pointing came from individual differences, and a method to classify the kind of point and derive its temporal parameters is discussed. These results and methods improve the pragmatism of virtual reality, making events appear more realistic by emphasizing temporal precision.

Putting context into context: sources of context and a proposed mechanism for linguistic negation

ABSTRACT Linguistic negation is often underspecified. The eagle is not in the sky does not specify a concrete meaning. For this reason, context is crucial, and can substantially mediate the meaning of a negated sentence. A core issue in the area of sentence processing and incorporation of context, is what information constitutes context, and how is it processed in the moment. The first study demonstrates the ambiguity of certain negated sentences without sufficient context, and a mousetracking study revealed greater deviations toward a lure during affirmative sentence processing compared to negated sentence processing. This counterintuitive result is predicted by a model of referent activation and motor responses in which referents are treated independently, compared to a model where affirmative and negated referents compete with one another. These findings are presented in a framework processing context via information from stored lexical-semantic representations (memory-mediated) or concurrent perceptual information (perceptually-mediated).