Domenica Bueti

The parietal cortex and the representation of time, space, number and other magnitudes

The development of sub-disciplines within cognitive neuroscience follows common sense categories such as language, audition, action, memory, emotion and perception among others. There are also well-established research programmes into temporal perception, spatial perception and mathematical cognition that also reflect the subjective impression of how experience is constructed. There is of course no reason why the brain should respect these common sense, text book divisions and, here, we discuss the contention that generalized magnitude processing is a more accurate conceptual description of how the brain deals with information about time, space, number and other dimensions. The roots of the case for linking magnitudes are based on the use to which magnitude information is put (action), the way in which we learn about magnitudes (ontogeny), shared properties and locations of magnitude processing neurons, the effects of brain lesions and behavioural interference studies. Here, we assess this idea in the context of a theory of magnitude, which proposed common processing mechanisms of time, space, number and other dimensions.

Sensory and association cortex in time perception

The recent upsurge of interest in brain mechanisms of time perception is beginning to converge on some new starting points for investigating this long under studied aspect of our experience. In four experiments, we asked whether disruption of normal activity in human MT/V5 would interfere with temporal discrimination. Although clearly associated with both spatial and motion processing, MT/V5 has not yet been implicated in temporal processes. Following predictions from brain imaging studies that have shown the parietal cortex to be important in human time perception, we also asked whether disruption of either the left or right parietal cortex would interfere with time perception preferentially in the auditory or visual domain. The results show that the right posterior parietal cortex is important for timing of auditory and visual stimuli and that MT/V5 is necessary for timing only of visual events.

Encoding of Temporal Probabilities in the Human Brain

Anticipating the timing of future events is a necessary precursor to preparing actions and allocating resources to sensory processing. This requires elapsed time to be represented in the brain and used to predict the temporal probability of upcoming events. While neuropsychological, imaging, magnetic stimulation studies, and single-unit recordings implicate the role of higher parietal and motor-related areas in temporal estimation, the role of earlier, purely sensory structures remains more controversial. Here we demonstrate that the temporal probability of expected visual events is encoded not by a single area but by a wide network that importantly includes neuronal populations at the very earliest cortical stages of visual processing. Moreover, we show that activity in those areas changes dynamically in a manner that closely accords with temporal expectations.

Modality-independent role of the primary auditory cortex in time estimation.

It has been argued that both modality-specific and supramodal mechanisms dedicated to time perception underlie the estimation of interval durations. While it is generally assumed that early sensory areas are dedicated to modality-specific time estimation, we hypothesized that early sensory areas such as the primary visual cortex or the auditory cortex might be involved in time perception independently of the sensory modality of the input. To test this possibility, we examined whether disruption of the primary visual cortex or the auditory cortex would disrupt time estimation of auditory stimuli and visual stimuli using transcranial magnetic stimulation (TMS). We found that disruption of the auditory cortex impaired not only time estimation of auditory stimuli but also impaired that of visual stimuli to the same degree. This finding suggests a supramodal role of the auditory cortex in time perception. On the other hand, TMS over the primary visual cortex impaired performance only in visual time discrimination. These asymmetric contributions of the auditory and visual cortices in time perception may be explained by a superiority of the auditory cortex in temporal processing. Here, we propose that time is primarily encoded in the auditory system and that visual inputs are automatically encoded into an auditory representation in time discrimination tasks.

No inherent left and right side in human ‘mental number line’: evidence from right brain damage

Spatial reasoning has a relevant role in mathematics and helps daily computational activities. It is widely assumed that in cultures with left-to-right reading, numbers are organized along the mental equivalent of a ruler, the mental number line, with small magnitudes located to the left of larger ones. Patients with right brain damage can disregard smaller numbers while mentally setting the midpoint of number intervals. This has been interpreted as a sign of spatial neglect for numbers on the left side of the mental number line and taken as a strong argument for the intrinsic left-to-right organization of the mental number line. Here, we put forward the understanding of this cognitive disability by discovering that patients with right brain damage disregard smaller numbers both when these are mapped on the left side of the mental number line and on the right side of an imagined clock face. This shows that the right hemisphere supports the representation of small numerical magnitudes independently from their mapping on the left or the right side of a spatial-mental layout. In addition, the study of the anatomical correlates through voxel-based lesion-symptom mapping and the mapping of lesion peaks on the diffusion tensor imaging-based reconstruction of white matter pathways showed that the rightward bias in the imagined clock-face was correlated with lesions of high-level middle temporal visual areas that code stimuli in object-centred spatial coordinates, i.e. stimuli that, like a clock face, have an inherent left and right side. In contrast, bias towards higher numbers on the mental number line was linked to white matter damage in the frontal component of the parietal-frontal number network. These anatomical findings show that the human brain does not represent the mental number line as an object with an inherent left and right side. We conclude that the bias towards higher numbers in the mental bisection of number intervals does not depend on left side spatial, imagery or object-centred neglect and that it rather depends on disruption of an abstract non-spatial representation of small numerical magnitudes.

Learning about time: plastic changes and interindividual brain differences.

Learning the timing of rapidly changing sensory events is crucial to construct a reliable representation of the environment and to efficiently control behavior. The neurophysiological mechanisms underlying the learning of time are unknown. We used functional and structural magnetic resonance imaging to investigate neurophysiological changes and individual brain differences underlying the learning of time in the millisecond range. We found that the representation of a trained visual temporal interval was associated with functional and structural changes in a sensory-motor network including occipital, parietal, and insular cortices, plus the cerebellum. We show that both types of neurophysiological changes correlated with changes of performance accuracy and that activity and gray-matter volume of sensorimotor cortices predicted individual learning abilities. These findings represent neurophysiological evidence of functional and structural plasticity associated with the learning of time in humans and highlight the role of sensory-motor circuits in the perceptual representation of time in the millisecond range.

The Sensory Representation of Time

Time is embedded in many aspects of our sensory experience; sensory events unfold in time and often acquire particular meaning because of their specific temporal structure. The speed of a moving object, the words pronounced by a speaker and the tactile exploration of a texture, are all examples of temporally structured sensory experiences. Despite the ubiquitousness of the temporal dimension of our sensory experience, the understanding of the neural mechanisms underlying the temporal representation of sensory events, that is the capacity to estimate duration in milliseconds/seconds range, remains a controversial and complex issue. The controversy relates to the effective involvement of sensory-specific brain regions in the processing of temporal information. The complexity arises from the neurophysiological mechanisms underlying the representation of time in these areas and the functional interplay between sensory-specific and amodal temporal mechanisms (Harrington et al., 2011). The idea that we time sensory signals via a single “centralized” and “amodal” clock dominated the field of temporal cognition over the last 30 years. More recently the universality of timing mechanisms has been challenged by new theoretical positions and a growing body of empirical data (Buhusi and Meck, 2005). From a theoretical perspective the challenge comes from “distributed” timing models. This is a broad class of models, which – although different regarding the neurophysiological mechanisms proposed for time processing – collectively share the idea that we have multiple timing mechanisms “distributed” across brain areas or circuits; and that the engagement of each single mechanism depends on the psychophysical task, sensory modality, and lengths of temporal intervals (Ivry and Richardson, 2002; Durstewitz, 2003; Matell and Meck, 2004; Buonomano and Maass, 2009). The idea that sensory-specific timing mechanisms exist is supported by studies showing that the ability to discriminate temporal information depends on the modality of the signals. For example, temporal discrimination thresholds are lower for auditory compared to visual signal durations (Grondin, 1993; Grondin et al., 2005; Merchant et al., 2008); and the capacity to keep in memory multiple intervals improves if the temporal signals belong to different modalities and therefore rely on different memory resources (Gamache and Grondin, 2010). The existence of independent sensory-specific clocks is also suggested by the observation that the perceived duration of a sensory event can be distorted by modality-specific properties of the stimuli such as visual adaptation (Johnston et al., 2006; Ayhan et al., 2009), spatial, and temporal frequency (Kanai et al., 2006; Kaneko and Murakami, 2009); or by the observation that such distortions are limited to a single sensory domain, like in case of saccadic eye movements causing compression of the perceived duration of visual but not of auditory stimuli (Morrone et al., 2005; Burr et al., 2011). From the neurophysiological point of view, electrophysiological recordings in animals as well as neuroimaging and magnetic stimulation studies in humans suggest that both modality-specific and supramodal mechanisms underlie the estimation of temporal intervals (Ghose and Maunsell, 2002; Shuler and Bear, 2006; Bosco et al., 2008; Bueti et al., 2008b; Sadeghi et al., 2011). For example, it has been demonstrated that the extrastriate visual area MT/V5 is necessary for temporal discrimination of visual, but not of auditory durations (Bueti et al., 2008a) and that duration estimation to predict expected visual and auditory events involves secondary as well as primary visual and auditory cortices (Ghose and Maunsell, 2002; Shuler and Bear, 2006; Bueti and Macaluso, 2010; Bueti et al., 2010). Taken together these behavioral and neurophysiological data highlight the functional contribution of sensory-specific cortices and support the existence of modality-specific timing mechanisms. However, how temporal information is actually represented in these cortices and what is the neurophysiological mechanism behind it, remain unclear. A few interesting theoretical hypotheses have been advanced. “Intrinsic” timing models for example, describe time as a general and inherent property of neural dynamics. A consequence of this assumption is that any area of the brain is in principle able to encode time. Temporal computations according to these models rely on inherent temporal properties of neural networks like short-term synaptic plasticity [i.e., state-dependent networks (SDNs) model; Buonomano and Maass, 2009] or arise either from the overall magnitude of neural activity (Eagleman, 2008) or from the linear ramping of neuronal firing rate (Durstewitz, 2003; Reutimann et al., 2004). “Intrinsic models” of temporal coding are particularly suitable to describe the functional organization of sensory timing mechanisms because they assume that time is encoded by the same circuits encoding other stimulus properties such as color or motion in the visual modality. However the explanatory power of some of these models, like for example the SDNs model, is constrained to durations of a few hundred milliseconds (i.e., <500 ms; Buonomano et al., 2009; Spencer et al., 2009); this is indeed a strong limitation, given that most of the neurophysiological evidence in favor of modality-specific timing mechanisms deal with durations from hundreds of milliseconds to a few seconds. An alternative possibility is that temporal computations in sensory cortices engage wider and specialized temporal circuit (s), where time signals from sensory cortex are sent to “dedicated” timing areas where these signals are integrated and used to guide action for example (Coull et al., 2011). In this latter case the relationship between sensory-specific and sensory independent timing areas need to be elucidated. Many cortical (parietal, premotor, prefrontal, and insular cortices) and subcortical (basal ganglia and cerebellum) brain structures have indeed been implicated in the processing of temporal information independently from the sensory modality of the stimuli (see Spencer et al., 2003; Coull et al., 2004; Koch et al., 2008; Wiener et al., 2010 for a review; Wittmann et al., 2010). Although there is only a partial agreement regarding the relevance of all these structures to time processing, the challenge is now to explore whether these areas have dissociable or interchangeable/overlapping functional roles and therefore whether these areas support the same or different temporal mechanisms compared to sensory-specific areas. A very special case of multimodal timing area is represented by the auditory cortex, a sensory-specific area. It has been recently demonstrated indeed that the auditory cortex is important for temporal discrimination not only of auditory but also of somatosensory and visual stimuli (Bolognini et al., 2009; Kanai et al., 2011). The supramodal involvement of auditory areas in temporal tasks has been associated with a strategic use of auditory-based mental representations for time estimation (Franssen et al., 2006). An interesting hypothesis, suggested by Kanai and colleagues, is that given the dominance of the auditory system over vision in temporal tasks (Walker and Scott, 1981; Burr et al., 2009), visual information is converted into an auditory code for temporal computation(Kanai et al., 2011). This hypothesis is interesting because offers new insight into the relationship between visual and auditory timing systems and highlights a possible link between modality independent and modality-specific temporal mechanisms. It is therefore clear that the study of the functional architecture of sensory timing mechanisms poses a few more theoretical and experimental challenges. A few important questions are still open. It is, for example, unclear whether the organizational principles that apply to space also apply to time and whether the temporal dimension of visual stimuli is processed by the same or distinct networks compared to those for space. Is time coding in visual cortex retinotopic specific? Do we encode all possible temporal intervals at each retinotopic position? In which context do sensory-specific temporal mechanisms work? Is temporal information encoded in sensory cortices automatically or does it require explicit attention? Are sensory areas engaged only during duration encoding or are also active during working memory maintenance? The already complex scenario of the neural representation of time is getting even more intricate. From the idea of a single “amodal” mechanism we moved into the idea of multiple “modality-specific” and “modality independent” temporal mechanisms (Wiener et al., 2011). The challenge is now to find out the functional architecture of these mechanisms as well as the interaction between them. As a concluding remark, I would like to emphasize that the focus of the majority of studies exploring the neural correlates of temporal processing has been so far to identifying the key components of internal timing networks (i.e., the “where” of timing mechanisms). The result of this approach has been, for example, an exponential increase of the number of neuroimaging studies on this topic that has lead to a substantial disagreement regarding the structures that are relevant to time processing (Wiener et al., 2010 for a review). It is time to adopt new experimental approaches that pose more mechanistically oriented questions about the underlying timing mechanisms while at the same time attempting to link computational models and neurophysiology (Portugal et al., 2011).

The Role of Superior Temporal Cortex in Auditory Timing

Recently, there has been upsurge of interest in the neural mechanisms of time perception. A central question is whether the representation of time is distributed over brain regions as a function of stimulus modality, task and length of the duration used or whether it is centralized in a single specific and supramodal network. The answers seem to be converging on the former, and many areas not primarily considered as temporal processing areas remain to be investigated in the temporal domain. Here we asked whether the superior temporal gyrus, an auditory modality specific area, is involved in processing of auditory timing. Repetitive transcranial magnetic stimulation was applied over left and right superior temporal gyri while participants performed either a temporal or a frequency discrimination task of single tones. A significant decrease in performance accuracy was observed after stimulation of the right superior temporal gyrus, in addition to an increase in response uncertainty as measured by the Just Noticeable Difference. The results are specific to auditory temporal processing and performance on the frequency task was not affected. Our results further support the idea of distributed temporal processing and speak in favor of the existence of modality specific temporal regions in the human brain.

Auditory temporal expectations modulate activity in visual cortex

Temporal expectation is the ability to make predictions and to use temporal information to anticipate the occurrence of future events. This capacity is associated with highly efficient perceptual and motor behaviors. However, how cognitive systems use temporal information to optimize behavior and what brain structures are engaged during these processes remains largely unknown. Neurophysiological and recent neuroimaging data have suggested that temporal expectations modulate activity not only in parietal and motor-related frontal regions, but also in occipital visual cortex, when the expected stimulus is a simple visual object. Here we investigate crossmodal properties and category selectivity of temporal expectations examining activity in visual cortex during expectation of auditory stimuli (the sound of hand-clapping or of a hammer-hammering). We found that activity in occipital cortex changed over time, reflecting the subjects temporal expectations about the upcoming auditory event. This modulatory effect included extrastriate visual areas known to process body-parts and tools, despite these were never presented visually during the experiment. However activity in these areas was not specific for the expected sound category, but it was rather related to the overall probability of the auditory target to occur. We conclude that crossmodal associations can influence activity in sensory-specific visual areas in an anticipatory manner, consistent with temporal expectations affecting activity in a distributed system of motor-related and sensory-related brain regions.

Memory for Time Distinguishes between Perception and Action

Our experience of time is unlike that of other features of the sensory world such as colour, movement, touch, or sound because there is no unique receptor system through which it is received. However, since time can be perceived, remembered, estimated, and compared in a way analogous to other sensory experiences, it should perhaps be subject to some of the same architectures or principles that have advanced understanding in these other domains. By adapting a task designed to test visual memory within a perception/action framework we investigated whether memory for time is affected by the use to which temporal information is put. When remembering a visual or auditory duration for subsequent motor production, storage is biased by a delay of up to 8 s. When the same duration is remembered for subsequent perception, however, there is no such effect of delay on memory. The results suggest a distinction in temporal memory that parallels the perception/action dichotomy in vision.