Joost van de Weijer

The influence of calibration method and eye physiology on eyetracking data quality

Recording eye movement data with high quality is often a prerequisite for producing valid and replicable results and for drawing well-founded conclusions about the oculomotor system. Today, many aspects of data quality are often informally discussed among researchers but are very seldom measured, quantified, and reported. Here we systematically investigated how the calibration method, aspects of participants’ eye physiologies, the influences of recording time and gaze direction, and the experience of operators affect the quality of data recorded with a common tower-mounted, video-based eyetracker. We quantified accuracy, precision, and the amount of valid data, and found an increase in data quality when the participant indicated that he or she was looking at a calibration target, as compared to leaving this decision to the operator or the eyetracker software. Moreover, our results provide statistical evidence of how factors such as glasses, contact lenses, eye color, eyelashes, and mascara influence data quality. This method and the results provide eye movement researchers with an understanding of what is required to record high-quality data, as well as providing manufacturers with the knowledge to build better eyetrackers.

Are parrots poor at motor self-regulation or is the cylinder task poor at measuring it?

The ability to inhibit unproductive motor responses triggered by salient stimuli is a fundamental inhibitory skill. Such motor self-regulation is thought to underlie more complex cognitive mechanisms, like self-control. Recently, a large-scale study, comparing 36 species, found that absolute brain size best predicted competence in motor inhibition, with great apes as the best performers. This was challenged when three Corvus species (corvids) were found to parallel great apes despite having much smaller absolute brain sizes. However, new analyses suggest that it is the number of pallial neurons, and not absolute brain size per se, that correlates with levels of motor inhibition. Both studies used the cylinder task, a detour-reaching test where food is presented behind a transparent barrier. We tested four species from the order Psittaciformes (parrots) on this task. Like corvids, many parrots have relatively large brains, high numbers of pallial neurons, and solve challenging cognitive tasks. Nonetheless, parrots performed markedly worse than the Corvus species in the cylinder task and exhibited strong learning effects in performance and response times. Our results suggest either that parrots are poor at controlling their motor impulses, and hence that pallial neuronal numbers do not always correlate with such skills, or that the widely used cylinder task may not be a good measure of motor inhibition.

Antonym canonicity: Temporal and contextual manipulations

Previous research on antonyms has shown that some pairings form more felicitous couplings than others. Following up on that research, we conducted two semantic categorization experiments using Event Related Potentials to establish whether there are neurophysiological differences related to levels of antonym canonicity. In Experiment 1, the members of canonical antonym pairs (e.g. black-white), non-canonical antonym pairs (e.g. white-dark) and unrelated word pairs (e.g. bumpy-small) were presented in isolation separated either by a short (200 ms) or a long (800 ms) time interval. The canonical antonyms gave rise to significantly lower N400 amplitudes than both non-canonical antonyms and unrelated pairs, but no significant difference in N400 amplitudes for non-canonical and unrelated pairs was found. In Experiment 2, the same pairs were presented in a congruent context. Significant differences in N400 amplitudes across all three conditions were found, also between non-canonical antonyms and unrelated word pairs.

Coordination of Gaze and Speech in Communication Between Children With Hearing Impairment and Normal-Hearing Peers

PURPOSE To investigate gaze behavior during communication between children with hearing impairment (HI) and normal-hearing (NH) peers. METHOD Ten HI-NH and 10 NH-NH dyads performed a referential communication task requiring description of faces. During task performance, eye movements and speech were tracked. Using verbal event (questions, statements, back channeling, and silence) as the predictor variable, group characteristics in gaze behavior were expressed with Kaplan-Meier survival functions (estimating time to gaze-to-partner) and odds ratios (comparing number of verbal events with and without gaze-to-partner). Analyses compared the listeners in each dyad (HI: n = 10, mean age = 12;6 years, mean better ear pure-tone average = 33.0 dB HL; NH: n = 10, mean age = 13;7 years). RESULTS Log-rank tests revealed significant group differences in survival distributions for all verbal events, reflecting a higher probability of gaze to the partners face for participants with HI. Expressed as odds ratios (OR), participants with HI displayed greater odds for gaze-to-partner (ORs ranging between 1.2 and 2.1) during all verbal events. CONCLUSIONS The results show an increased probability for listeners with HI to gaze at the speakers face in association with verbal events. Several explanations for the finding are possible, and implications for further research are discussed.

Semantic profiles of antonymic adjectives in discourse

Abstract This study has two goals: First, to give an account of the semantic organization of individually used antonymic adjectives in discourse and second, based on those findings and previous work on antonymic meanings, to contribute to a comprehensive theoretical account of their representation within the framework of Cognitive Linguistics. The hypothesis is that the members of the pairs are used in the same contexts and in the same type of constructions, not only when they co-occur and are used to express binary opposition as shown in previous studies, but also when they do not. The manually coded corpus data from the BNC are analyzed along four semantic parameters: (i) the configuration of the adjectives in terms of gradability, (ii) the way they modify the nominal meanings, i.e., attributively or predicatively (iii) the meaning type of the modified nouns, and (iv) the status of the constructions with respect to whether their meanings are what we refer to as “basic”, metaphorical or metonymical. Correspondence analysis technique is used to identify similarities and differences on the basis of the totality of the data. As predicted, our findings confirm a high degree of pairwise similarity – but also some differences. On the basis of these results, it can be argued that the long-standing controversy within Structuralism between proponents of the co-occurrence hypothesis and the substitutability hypothesis in antonym research is a non-issue.

Impact of cognitive and linguistic ability on gaze behavior in children with hearing impairment

In order to explore verbal–nonverbal integration, we investigated the influence of cognitive and linguistic ability on gaze behavior during spoken language conversation between children with mild-to-moderate hearing impairment (HI) and normal-hearing (NH) peers. Ten HI–NH and 10 NH-NH dyads performed a referential communication task requiring description of faces. During task performance, eye movements and speech were tracked. Cox proportional hazards regression was used to model associations between performance on cognitive and linguistic tasks and the probability of gaze to the conversational partner’s face. Analyses compare the listeners in each dyad (HI: n = 10, mean age = 12; 6 years, SD = 2; 0, mean better ear pure-tone average 33.0 dB HL, SD = 7.8; NH: n = 10, mean age = 13; 7 years, SD = 1; 11). Group differences in gaze behavior – with HI gazing more to the conversational partner than NH – remained significant despite adjustment for ability on receptive grammar, expressive vocabulary, and complex working memory. Adjustment for phonological short term memory, as measured by non-word repetition, removed group differences, revealing an interaction between group membership and non-word repetition ability. Stratified analysis showed a twofold increase of the probability of gaze-to-partner for HI with low phonological short term memory capacity, and a decreased probability for HI with high capacity, as compared to NH peers. The results revealed differences in gaze behavior attributable to performance on a phonological short term memory task. Participants with HI and low phonological short term memory capacity showed a doubled probability of gaze to the conversational partner, indicative of a visual bias. The results stress the need to look beyond the HI in diagnostics and intervention. Acknowledgment of the finding requires clinical assessment of children with HI to be supported by tasks tapping phonological processing.

Voice onset time and global foreign accent in German-French simultaneous bilinguals during adulthood

Aims and objectives: In this study, we investigated crosslinguistic influence in the phonetic systems of simultaneous bilinguals (2L1s) during adulthood. Methodology: Specifically, we analyzed the voice onset time (VOT) of the voiceless stop /k/ in the spontaneous speech of 14 German–French bilinguals who grew up in France or Germany. We looked at both languages, first comparing the groups, second comparing their VOT to their global accent. Data and analysis: The material consisted of interviews, lasting for about half an hour. Findings/conclusions: Most 2L1s showed distinct VOT-ranges in their two languages, even if they were perceived to have a foreign accent in the minority language of their childhood environment. We conclude that the phonetic systems of 2L1s remain separate and stable throughout the lifespan. However, the 2L1s from France had significantly shorter VOTs in German than the 2L1s from Germany, and their speech was overall more accented. These findings are discussed with respect to the role of intra- and extra-linguistic factors. Originality: Our study adds a new perspective to existing VOT studies of bilinguals by using naturalistic speech data and by comparing two groups of 2L1s who have the same language combination but grew up in different countries, which allows us to evaluate the impact of their childhood environment on VOT development. Significance/implications: Language exposure during childhood seems to be beneficial for pronunciation during adulthood.

The Impact of Visual and Auditory Cues in Age Estimation

Several factors determine the ease and accuracy with which we can estimate a speakers age. The question addressed in our study is to what extent visual and auditory cues compete with each other. We investigated this question in a series of five related experiments. In the first four experiments, subjects estimated the age of 14 female speakers, either from still pictures, an audio recording, a video recording without sound, or a video recording with sound. The results from the first four experiments were used in the fifth experiment, to combine the speakers with new voices, so that there was a discrepancy in how old the speaker looked and how old she sounded. The estimated ages of these dubbed videos were not significantly different from those of the original videos, suggesting that voice has little impact on the estimation of age when visual cues are available.

Does the Kuleshov Effect Really Exist? Revisiting a Classic Film Experiment on Facial Expressions and Emotional Contexts:

According to film mythology, the Soviet filmmaker Lev Kuleshov conducted an experiment in which he combined a close-up of an actor’s neutral face with three different emotional contexts: happiness, sadness, and hunger. The viewers of the three film sequences reportedly perceived the actor’s face as expressing an emotion congruent with the given context. It is not clear, however, whether or not the so-called “Kuleshov effect” really exists. The original film footage is lost and recent attempts at replication have produced either conflicting or unreliable results. The current paper describes an attempt to replicate Kuleshov’s original experiment using an improved experimental design. In a behavioral and eye tracking study, 36 participants were each presented with 24 film sequences of neutral faces across six emotional conditions. For each film sequence, the participants were asked to evaluate the emotion of the target person in terms of valence, arousal, and category. The participants’ eye movements were recorded throughout. The results suggest that some sort of Kuleshov effect does in fact exist. For each emotional condition, the participants tended to choose the appropriate category more frequently than the alternative options, while the answers to the valence and arousal questions also went in the expected directions. The eye tracking data showed how the participants attended to different regions of the target person’s face (in light of the intermediate context), but did not reveal the expected differences between the emotional conditions.

Evaluative polarity of antonyms

This study investigates speakers’ assessment of the evaluative polarity of the members of eight antonym pairs, e.g., fast–slow and warm–cold, that are not inherently evaluative, unlike antonyms such as good–bad, ugly–beautiful. The contentful structures foregrounded by fast–slow and warm–cold are SPEED and TEMPERATURE, repectively, but the properties that they evoke may also be profiled against a dimension of positive and negative polarity. In this article we adapt the Implicit Association Test (IAT) to measure whether speakers in fact associate such antonym pairs with positivity and negativity, and if they do, which is positive and which is negative. The results of the experiments show clear and consistent polarity patterns across the antonym pairs under investigation, i.e. one of the members of a pair of antonyms is more readily associated with negativity and the other with positivity.