Sung-Joo Lim

Studying the effects of personalized language and worked examples in the context of a web-based intelligent tutor

Previous studies have demonstrated the learning benefit of personalized language and worked examples. However, previous investigators have primarily been interested in how these interventions support students as they problem solve with no other cognitive support. We hypothesized that personalized language added to a web-based intelligent tutor and worked examples provided as complements to the tutor would improve student (e-)learning. However, in a 2 x 2 factorial study, we found that personalization and worked examples had no significant effects on learning. On the other hand, there was a significant difference between the pretest and posttest across all conditions, suggesting that the online intelligent tutor present in all conditions did make a difference in learning. We conjecture why personalization and, especially, the worked examples did not have the hypothesized effect in this preliminary experiment, and discuss a new study we have begun to further investigate these effects.

Evidence for Cerebellar Contributions to Adaptive Plasticity in Speech Perception

Human speech perception rapidly adapts to maintain comprehension under adverse listening conditions. For example, with exposure listeners can adapt to heavily accented speech produced by a non-native speaker. Outside the domain of speech perception, adaptive changes in sensory and motor processing have been attributed to cerebellar functions. The present functional magnetic resonance imaging study investigates whether adaptation in speech perception also involves the cerebellum. Acoustic stimuli were distorted using a vocoding plus spectral-shift manipulation and presented in a word recognition task. Regions in the cerebellum that showed differences before versus after adaptation were identified, and the relationship between activity during adaptation and subsequent behavioral improvements was examined. These analyses implicated the right Crus I region of the cerebellum in adaptive changes in speech perception. A functional correlation analysis with the right Crus I as a seed region probed for cerebral cortical regions with covarying hemodynamic responses during the adaptation period. The results provided evidence of a functional network between the cerebellum and language-related regions in the temporal and parietal lobes of the cerebral cortex. Consistent with known cerebellar contributions to sensorimotor adaptation, cerebro-cerebellar interactions may support supervised learning mechanisms that rely on sensory prediction error signals in speech perception.

How may the basal ganglia contribute to auditory categorization and speech perception

Listeners must accomplish two complementary perceptual feats in extracting a message from speech. They must discriminate linguistically-relevant acoustic variability and generalize across irrelevant variability. Said another way, they must categorize speech. Since the mapping of acoustic variability is language-specific, these categories must be learned from experience. Thus, understanding how, in general, the auditory system acquires and represents categories can inform us about the toolbox of mechanisms available to speech perception. This perspective invites consideration of findings from cognitive neuroscience literatures outside of the speech domain as a means of constraining models of speech perception. Although neurobiological models of speech perception have mainly focused on cerebral cortex, research outside the speech domain is consistent with the possibility of significant subcortical contributions in category learning. Here, we review the functional role of one such structure, the basal ganglia. We examine research from animal electrophysiology, human neuroimaging, and behavior to consider characteristics of basal ganglia processing that may be advantageous for speech category learning. We also present emerging evidence for a direct role for basal ganglia in learning auditory categories in a complex, naturalistic task intended to model the incidental manner in which speech categories are acquired. To conclude, we highlight new research questions that arise in incorporating the broader neuroscience research literature in modeling speech perception, and suggest how understanding contributions of the basal ganglia can inform attempts to optimize training protocols for learning non-native speech categories in adulthood.

When Is Assistance Helpful to Learning? Results in Combining Worked Examples and Intelligent Tutoring

When should instruction provide or withhold assistance? In three empirical studies, we have investigated whether worked examples, a high-assistance approach, studied in conjunction with tutored problems to be solved, a mid-level assistance approach, can lead to better learning. Contrary to prior results with untutoredproblem solving, a low-assistance approach, we found that worked examples alternating with isomorphic tutored problems did not produce more learning gains than tutored problems alone. However, the examples group across the three studies learned more efficiently than the tutored-alone group. Our studies, in conjunction with past studies, suggest that mid-level assistance leads to better learning than either lower or higher level assistance. However, while our results are illuminating, more work is needed to develop predictive theory for what combinations of assistance yield the most effective and efficient learning.

Discovering functional units in continuous speech.

Language learning requires that listeners discover acoustically variable functional units like phonetic categories and words from an unfamiliar, continuous acoustic stream. Although many category learning studies have examined how listeners learn to generalize across the acoustic variability inherent in the signals that convey the functional units of language, these studies have tended to focus upon category learning across isolated sound exemplars. However, continuous input presents many additional learning challenges that may impact category learning. Listeners may not know the timescale of the functional unit, its relative position in the continuous input, or its relationship to other evolving input regularities. Moving laboratory-based studies of isolated category exemplars toward more natural input is important to modeling language learning, but very little is known about how listeners discover categories embedded in continuous sound. In 3 experiments, adult participants heard acoustically variable sound category instances embedded in acoustically variable and unfamiliar sound streams within a video game task. This task was inherently rich in multisensory regularities with the to-be-learned categories and likely to engage procedural learning without requiring explicit categorization, segmentation, or even attention to the sounds. After 100 min of game play, participants categorized familiar sound streams in which target words were embedded and generalized this learning to novel streams as well as isolated instances of the target words. The findings demonstrate that even without a priori knowledge, listeners can discover input regularities that have the best predictive control over the environment for both non-native speech and nonspeech signals, emphasizing the generality of the learning.

Dopaminergic modulation of hemodynamic signal variability and the functional connectome during cognitive performance

&NA; Dopamine underlies important aspects of cognition, and has been suggested to boost cognitive performance. However, how dopamine modulates the large‐scale cortical dynamics during cognitive performance has remained elusive. Using functional MRI during a working memory task in healthy young human listeners, we investigated the effect of levodopa (l‐dopa) on two aspects of cortical dynamics, blood oxygen‐level‐dependent (BOLD) signal variability and the functional connectome of large‐scale cortical networks. We here show that enhanced dopaminergic signaling modulates the two potentially interrelated aspects of large‐scale cortical dynamics during cognitive performance, and the degree of these modulations is able to explain inter‐individual differences in l‐dopa‐induced behavioral benefits. Relative to placebo, l‐dopa increased BOLD signal variability in task‐relevant temporal, inferior frontal, parietal and cingulate regions. On the connectome level, however, l‐dopa diminished functional integration across temporal and cingulo‐opercular regions. This hypo‐integration was expressed as a reduction in network efficiency and modularity in more than two thirds of the participants and to different degrees. Hypo‐integration co‐occurred with relative hyper‐connectivity in paracentral lobule and precuneus, as well as posterior putamen. Both, l‐dopa‐induced BOLD signal variability modulation and functional connectome modulations proved predictive of an individuals l‐dopa‐induced benefits in behavioral performance, namely response speed and perceptual sensitivity. Lastly, l‐dopa‐induced modulations of BOLD signal variability were correlated with l‐dopa‐induced modulation of nodal connectivity and network efficiency. Our findings underline the role of dopamine in maintaining the dynamic range of, and communication between, cortical systems, and their explanatory power for inter‐individual differences in benefits from dopamine during cognitive performance. HighlightsDopamine (DA) increases BOLD signal variability in an auditory working memory task.DA increase induces hyper‐connectivity within the functional connectome.DA increase diminishes network integration of temporal and cingulo‐opercular regions.Signal variability and connectome changes correlate with behavioral gains from DA.DA maintains cortical information processing by tuning signal and network dynamics.

Supporting research into sound and speech learning through a configurable computer game

Cognitive neuroscientists studying sound and speech learning have successfully used videogames as a research vehicle. Neuroscientists and game developers worked together to produce a game built to entice participants to longer periods of play, while enabling researchers to easily configure presentation parameters in support of future studies. A space-themed game polished through the use of shaders and a radial cannon shooting mechanic is detailed, along with lessons learned from iterative playtesting. A preliminary study indicates the games effectiveness for implicit learning of sounds. The template by which this game can be tuned to explore language learning is presented, with suggestions for future investigations into the tradeoffs between learning transfer and game appeal.

Learning acoustically complex word‐like units within a video‐game training paradigm.

Over the course of language development, infants learn native speech categories and word boundaries from speech input. Although speech category learning and word segmentation learning occur in parallel, most investigations have focused on one, assuming somewhat mature develop of the other. To investigate the extent to which listeners can simultaneously solve the categorization and segmentation learning challenges, we created an artificial, non‐linguistic stimulus space that modeled the acoustic complexities of natural speech by recording a single talker’s multiple utterances of a set of sentences containing four keywords. There was acoustic variability across utterances, presenting a categorization challenge. The keywords were embedded in continuous speech, presenting a segmentation challenge. Sentences were spectrally rotated, rendering them wholly unintelligible, and presented within a video‐game training paradigm that does not rely upon explicit feedback and yet is effective in training non‐speech and ...

Investigating non‐native category learning using a video‐game‐based training paradigm.

Adults have difficulty learning non‐native speech categories, presenting an opportunity to study adult learning and plasticity with non‐native speech categorization. Long‐term training within laboratory‐based response‐feedback paradigms has produced modest non‐native category learning in previous studies. The current study investigates the effectiveness of a video‐game‐based categorization training paradigm, found to be effective in learning novel nonspeech auditory categories [W. Holt, (2005)], to train native Japanese adults to categorize English /r/ and /l/. This approach emphasizes functional associations between sound categories and players’ responses to video‐game characters rather than overt phonetic categorization. Although categorization is not explicit in the game it is helpful to overall performance, providing a functional and perhaps more ecologically valid training signal than traditional feedback in standard laboratory training procedures. Japanese participants who played the game for about ...

The Benefit of Attention-to-Memory Depends on the Interplay of Memory Capacity and Memory Load

Humans can be cued to attend to an item in memory, which facilitates and enhances the perceptual precision in recalling this item. Here, we demonstrate that this facilitating effect of attention-to-memory hinges on the overall degree of memory load. The benefit an individual draws from attention-to-memory depends on her overall working memory performance, measured as sensitivity (d′) in a retroactive cue (retro-cue) pitch discrimination task. While listeners maintained 2, 4, or 6 auditory syllables in memory, we provided valid or neutral retro-cues to direct listeners’ attention to one, to-be-probed syllable in memory. Participants’ overall memory performance (i.e., perceptual sensitivity d′) was relatively unaffected by the presence of valid retro-cues across memory loads. However, a more fine-grained analysis using psychophysical modeling shows that valid retro-cues elicited faster pitch-change judgments and improved perceptual precision. Importantly, as memory load increased, listeners’ overall working memory performance correlated with inter-individual differences in the degree to which precision improved (r = 0.39, p = 0.029). Under high load, individuals with low working memory profited least from attention-to-memory. Our results demonstrate that retrospective attention enhances perceptual precision of attended items in memory but listeners’ optimal use of informative cues depends on their overall memory abilities.