Lionel Granjon

Modelling Spatio-Temporal Saliency to Predict Gaze Direction for Short Videos

This paper presents a spatio-temporal saliency model that predicts eye movement during video free viewing. This model is inspired by the biology of the first steps of the human visual system. The model extracts two signals from video stream corresponding to the two main outputs of the retina: parvocellular and magnocellular. Then, both signals are split into elementary feature maps by cortical-like filters. These feature maps are used to form two saliency maps: a static and a dynamic one. These maps are then fused into a spatio-temporal saliency map. The model is evaluated by comparing the salient areas of each frame predicted by the spatio-temporal saliency map to the eye positions of different subjects during a free video viewing experiment with a large database (17000 frames). In parallel, the static and the dynamic pathways are analyzed to understand what is more or less salient and for what type of videos our model is a good or a poor predictor of eye movement.

Neural correlates of cued recall in young and older adults: an event-related potential study.

This experiment investigated age differences in electrophysiological correlates of retrieval success in a word-stem cued recall task. Young adults (M±SD: 21.4 years±1.9) performed this memory task more accurately than older participants (M±SD: 65.1 years±3.3). Robust event-related brain potential (ERP) old/new effects were identified in both age groups. The main age differences were observed in latency and lateralization of ERP effects. Young adults exhibited a parietal effect that became focused over left parietal electrodes, whereas no asymmetry was observed in older adults. Moreover, ERP effects were more delayed in the older group. Overall, these findings provide some evidence of the reduction of processing speed during aging and suggest that young and older adults may recruit distinct cerebral patterns during episodic cued recall.

The amount of retrieval support modulates age effects on episodic memory: Evidence from event-related potentials

This experiment was designed to explore the impact of age and amount of retrieval support on episodic memory and its electrophysiological correlates. Event-related potentials (ERPs) were recorded while young and older participants performed a word-stem cued-recall task in a low-support condition (LSC) in which the stem was composed of three letters, and a high-support condition (HSC) in which the cue consisted of four letters. Behavioral analyses showed that recall in the older group was less accurate than in the young group in the LSC, but no age differences were observed in the HSC. In the LSC, old/new ERP effects at frontal and parietal sites were later and less sustained for the older adults. Furthermore, the parietal old/new effect was symmetrically distributed for older adults, whereas it was predominant over the left hemisphere for their younger counterparts. In addition, young participants demonstrated early and long-lasting frontal and parietal effects in the HSC but with predominance over the right hemisphere, whereas the older adults exhibited a frontal effect and an early and long-lasting parietal effect becoming predominant over the left hemisphere. No age differences in the time course of the parietal old/new effect were observed in this more supportive condition. In addition, in the last period, the left parietal effect was greater for the older group. This study suggests that episodic memory performance and ERP correlates of recall processes are more similar between young and older adults when increased support is provided at retrieval.

Rushes summarization by IRIM consortium: redundancy removal and multi-feature fusion

In this paper, we present the first participation of a consortium of French laboratories, IRIM, to the TRECVID 2008 BBC Rushes Summarization task. Our approach resorts to video skimming. We propose two methods to reduce redundancy, as rushes include several takes of scenes. We also take into account low and mid-level semantic features in an ad-hoc fusion method in order to retain only significant content

Which limb is it? Responses to vibrotactile stimulation in early infancy

This study focuses on how the body schema develops during the first months of life, by investigating infants’ motor responses to localized vibrotactile stimulation on their limbs. Vibrotactile stimulation was provided by small buzzers that were attached to the infants’ four limbs one at a time. Four age groups were compared cross‐sectionally (3‐, 4‐, 5‐, and 6‐month‐olds). We show that before they actually reach for the buzzer, which, according to previous studies, occurs around 7–8 months of age, infants demonstrate emerging knowledge about their bodys configuration by producing specific movement patterns associated with the stimulated body area. At 3 months, infants responded with an increase in general activity when the buzzer was placed on the body, independently of the vibrators location. Differentiated topographical awareness of the body seemed to appear around 5 months, with specific responses resulting from stimulation of the hands emerging first, followed by the differentiation of movement patterns associated with the stimulation of the feet. Qualitative analyses revealed specific movement types reliably associated with each stimulated location by 6 months of age, possibly preparing infants’ ability to actually reach for the vibrating target. We discuss this result in relation to newborns’ ability to learn specific movement patterns through intersensory contingency. Statement of contribution what is already known on infants’ sensorimotor knowledge about their own bodies 3‐month‐olds readily learn to produce specific limb movements to obtain a desired effect (movement of a mobile). infants detect temporal and spatial correspondences between events involving their own body and visual events. what the present study adds until 4–5 months of age, infants mostly produce general motor responses to localized touch. this is because in the present study, infants could not rely on immediate contingent feedback. we propose a cephalocaudal developmental trend of topographic differentiation of body areas.

How should we test infant handedness

ABSTRACT As soon as infants grasp objects, they exhibit some degree of hand preference. Although all studies agree that the number of non-lateralized infants remains high during the first months of life, there is no consensus on the percentage of right- and left-handed infants. Reasons might be the different formulae used to calculate an handedness index, the basis on which handedness categories are distinguished, and the number of trials per session. In this study we aimed to provide a valid method of testing handedness, reliable without being lengthy so as to combine it with other evaluations. We tested 46 infants at 9 and 11 months by giving them 34 trials. We compared the results using 2 of the most used formulae, including the bimanual grasps or not, and considering the first 5, 10, 15, 20, 26, 31, or all 34 trials. The results show that different formulae do not give significantly different results as long as bimanual grasps are included. The number of trials is important: 15 trials are necessary for the reliability of the classification into handedness categories. Giving more trials does not provide substantial additional information.

The effect of connectives on the selection of arguments: implicit consequentiality bias for the connective "but".

Recent studies about the implicit causality of inter-personal verbs showed a symmetric implicit consequentiality bias for psychological verbs. This symmetry is less clear for action verbs because the verbs assigning the implicit cause to the object argument (e.g. “Peter protected John because he was in danger.”) tend to assign the implicit consequence to the same argument (e.g. “Peter protected John so he was not hurt.”). We replicated this result by comparing continuations of inter-personal events followed by a causal connective “because” or a consequence connective “so”. Moreover, we found similar results when the consequence connective was replaced by a contrastive connective “but”. This result was confirmed in a second experiment where we measured the time required to imagine a consistent continuation for a fragment finishing with “but s/he ...”. The results were consistent with a contrastive connective introducing a denial of a consequence of the previous event. The results were consistent with a model suggesting that thematic roles and connectives can predict preferred co-reference relations.

How different kinds of sound in videos can influence gaze

This paper presents an analysis of the effect of thirteen different kinds of sound on visual gaze when looking freely at videos to help to predict eye positions. First, an audio-visual experiment was designed with two groups of participants, with audio-visual (AV) and visual (V) conditions, to test the sound effect. Then, an audio experiment was designed to validate the classification of sound we proposed. We observed that the sound effect is different depending on the kind of sound, and that the classes with human voice (speech, singer, human noise and singers) have the greatest effect. Finally, a comparison of eye positions with a visual saliency model was carried out, which proves that adding sound to video decreases the accuracy of prediction of the visual saliency model.

Temporal expectancies driven by self- and externally generated rhythms

&NA; The dynamic attending theory proposes that rhythms entrain periodic fluctuations of attention which modulate the gain of sensory input. However, temporal expectancies can also be driven by the mere passage of time (foreperiod effect). It is currently unknown how these two types of temporal expectancy relate to each other, i.e. whether they work in parallel and have distinguishable neural signatures. The current research addresses this issue. Participants either tapped a 1 Hz rhythm (active task) or were passively presented with the same rhythm using tactile stimulators (passive task). Based on this rhythm an auditory target was then presented early, in synchrony, or late. Behavioural results were in line with the dynamic attending theory as RTs were faster for in‐ compared to out‐of‐synchrony targets. Electrophysiological results suggested self‐generated and externally induced rhythms to entrain neural oscillations in the delta frequency band. Auditory ERPs showed evidence of two distinct temporal expectancy processes. Both tasks demonstrated a pattern which followed a linear foreperiod effect. In the active task, however, we also observed an ERP effect consistent with the dynamic attending theory. This study shows that temporal expectancies generated by a rhythm and expectancy generated by the mere passage of time can work in parallel and sheds light on how these mechanisms are implemented in the brain. HighlightsTemporal expectancies can be driven by rhythms or by the mere passage of time.We conceived a paradigm using EEG that tests for both these forms of expectancy.We compared self‐generated and externally generated rhythms.We found evidence for both expectancy processes working in parallel.

Second Language Ability and Emotional Prosody Perception

The present study examines the effect of language experience on vocal emotion perception in a second language. Native speakers of French with varying levels of self-reported English ability were asked to identify emotions from vocal expressions produced by American actors in a forced-choice task, and to rate their pleasantness, power, alertness and intensity on continuous scales. Stimuli included emotionally expressive English speech (emotional prosody) and non-linguistic vocalizations (affect bursts), and a baseline condition with Swiss-French pseudo-speech. Results revealed effects of English ability on the recognition of emotions in English speech but not in non-linguistic vocalizations. Specifically, higher English ability was associated with less accurate identification of positive emotions, but not with the interpretation of negative emotions. Moreover, higher English ability was associated with lower ratings of pleasantness and power, again only for emotional prosody. This suggests that second language skills may sometimes interfere with emotion recognition from speech prosody, particularly for positive emotions.