Jill Fain Lehman

Goals and Learning in Microworlds

We explored the consequences for learning through interaction with an educational microworld called Electric Field Hockey (EFH). Like many microworlds, EFH is intended to help students develop a qualitative understanding of the target domain, in this case, the physics of electrical interactions. Through the development and use of a computer model that learns to play EFH, we analyzed the knowledge the model acquired as it applied the game-oriented strategies we observed physics students using. Through learning-by-doing on the standard sequence of tasks, the model substantially improved its EFH playing ability; however, it did so without acquiring any new qualitative physics knowledge. This surprising result led to an experiment that compared students’ use of EFH with standard-goal tasks against two alternative instructional conditions, specific-path and no-goal, each justified from a different learning theory. Students in the standard-goal condition learned less qualitative physics than did those in the two alternative conditions, which was consistent with the model. The implication for instructional practice is that careful selection and analysis of the tasks that frame microworld use is essential if these programs are to lead to the learning outcomes imagined for them. Theoretically, these results suggest a new interpretation for numerous empirical findings on the effectiveness of no-goal instructional tasks. The standing ‘‘reduced cognitive load’’ interpretation is contradicted by the success of the specific-path condition, and we offer an alternative knowledge-dependent interpretation.

Feel Effects: Enriching Storytelling with Haptic Feedback

Despite a long history of use in communication, haptic feedback is a relatively new addition to the toolbox of special effects. Unlike artists who use sound or vision, haptic designers cannot simply access libraries of effects that map cleanly to media content, and they lack even guiding principles for creating such effects. In this article, we make progress toward both capabilities: we generate a foundational library of usable haptic vocabulary and do so with a methodology that allows ongoing additions to the library in a principled and effective way. We define a feel effect as an explicit pairing between a meaningful linguistic phrase and a rendered haptic pattern. Our initial experiment demonstrates that users who have only their intrinsic language capacities, and no haptic expertise, can generate a core set of feel effects that lend themselves via semantic inference to the design of additional effects. The resulting collection of more than 40 effects covers a wide range of situations (including precipitation, animal locomotion, striking, and pulsating events) and is empirically shown to produce the named sensation for the majority of our test users in a second experiment. Our experiments demonstrate a unique and systematic approach to designing a vocabulary of haptic sensations that are related in both the semantic and parametric spaces.

Channel selection based on multichannel cross-correlation coefficients for distant speech recognition

In theory, beamforming performance can be improved by using as many microphones as possible, but in practice it has been shown that using all possible channels does not always improve speech recognition performance [1, 2, 3, 4, 5]. In this work, we present a new channel selection method in order to increase the computational efficiency of beamforming for distant speech recognition (DSR) without sacrficing performance.

FeelSleeve : Haptic Feedback to Enhance Early Reading

Engaging children with traditional approaches in education, especially reading, grows ever more difficult in the face of their attachment to tablets and computer games. We explore the possibility of making the story reading experience more interesting and memorable for children using haptic augmentation. In this paper, we present FeelSleeve, an interface that allows children to feel story events in their hands while they are reading on a mobile device. FeelSleeve uses transducers and audio output from the tablet within a gloved attachment to create vibratory effects that are meaningfully related to story content. We describe a study investigating whether embedding such haptic feedback into stories enhances reading for six to nine year olds. Our results indicate that story events accompanied by haptic feedback are better comprehended and appear to be more salient in memory. These results provide evidence that haptic effects have the potential to improve childrens reading experience and make it more memorable.

Is He Being Bad? Social and Language Brain Networks during Social Judgment in Children with Autism

Individuals with autism often violate social rules and have lower accuracy in identifying and explaining inappropriate social behavior. Twelve children with autism (AD) and thirteen children with typical development (TD) participated in this fMRI study of the neurofunctional basis of social judgment. Participants indicated in which of two pictures a boy was being bad (Social condition) or which of two pictures was outdoors (Physical condition). In the within-group Social–Physical comparison, TD children used components of mentalizing and language networks [bilateral inferior frontal gyrus (IFG), bilateral medial prefrontal cortex (mPFC), and bilateral posterior superior temporal sulcus (pSTS)], whereas AD children used a network that was primarily right IFG and bilateral pSTS, suggesting reduced use of social and language networks during this social judgment task. A direct group comparison on the Social–Physical contrast showed that the TD group had greater mPFC, bilateral IFG, and left superior temporal pole activity than the AD group. No regions were more active in the AD group than in the group with TD in this comparison. Both groups successfully performed the task, which required minimal language. The groups also performed similarly on eyetracking measures, indicating that the activation results probably reflect the use of a more basic strategy by the autism group rather than performance disparities. Even though language was unnecessary, the children with TD recruited language areas during the social task, suggesting automatic encoding of their knowledge into language; however, this was not the case for the children with autism. These findings support behavioral research indicating that, whereas children with autism may recognize socially inappropriate behavior, they have difficulty using spoken language to explain why it is inappropriate. The fMRI results indicate that AD children may not automatically use language to encode their social understanding, making expression and generalization of this knowledge more difficult.

Toward the use of speech and natural language technology in intervention for a language-disordered population

We describe the design of Simone Says an interactive soft- ware environment for language remediation that brings to- gether research in speech recognition, natural language pro- cessing and computer-aided instruction. The underlying technology for the implementation and the system?s even- tual evaluation are also discussed.

An Integrated Discourse Recipe-Based Model for Task-Oriented Dialogue

We present an integrated discourse recipe-based model (DRM) for dialogue generation and interpretation. Discourse recipes are generalizations of discourse plans. The DRM has been implemented as part of a conversational agent that supports task-oriented dialogue between human and artificial pilots. The design of the DRM has been strongly influenced by its implementation in the Soar cognitive architecture. In the DRM, discourse recipes are acquired as a side effect of dialogue planning. The discourse recipes can be used for generation and interpretation in future situations in place of planning and reasoning from first principles. We describe the motivation for a discourse recipe-based approach and present the design of the DRM.

FeelCraft: crafting tactile experiences for media using a feel effect library

FeelCraft is a media plugin that monitors events and states in the media and associates them with expressive tactile content using a library of feel effects (FEs). A feel effect (FE) is a user-defined haptic pattern that, by virtue of its connection to a meaningful event, generates dynamic and expressive effects on the users body. We compiled a library of more than fifty FEs associated with common events in games, movies, storybooks, etc., and used them in a sandbox-type gaming platform. The FeelCraft plugin allows a game designer to quickly generate haptic effects, associate them to events in the game, play them back for testing, save them and/or broadcast them to other users to feel the same haptic experience. Our demonstration shows an interactive procedure for authoring haptic media content using the FE library, playing it back during interactions in the game, and broadcasting it to a group of guests.

Real-time natural language generation in NL-Soar

NL-Soar is a computer system that performs language comprehension and generation within the framework of the Soar architecture [New90]. NL-Soar provides language capabilities for systems working in a real-time environment. Responding in real time to changing situations requires a flexible way to shift control between language and task operations. To provide this flexibility, NL-Soar organizes generation as a sequence of incremental steps that can be interleaved with task actions as the situation requires. This capability has been demonstrated via the integration of NL-Soar with two different independently-developed Soar-based systems.

Combining multiple knowledge sources in an integrated intelligent system

Using a stratified approach to system design embodied in Soar, multiple knowledge sources are integrated to implement systems performing different tasks: natural-language comprehension, production scheduling, and algorithm design. These three systems demonstrate that architectural mechanisms can play a key role in constructing systems to perform difficult knowledge-intensive tasks. Basic Soar principles are reviewed, and it is noted that Soar mechanisms reduce both design-time and runtime overhead associated with knowledge integration.<<ETX>>