Gwen A. Frishkoff

Automatic Question Generation for Vocabulary Assessment

In the REAP system, users are automatically provided with texts to read targeted to their individual reading levels. To find appropriate texts, the users vocabulary knowledge must be assessed. We describe an approach to automatically generating questions for vocabulary assessment. Traditionally, these assessments have been hand-written. Using data from WordNet, we generate 6 types of vocabulary questions. They can have several forms, including wordbank and multiple-choice. We present experimental results that suggest that these automatically-generated questions give a measure of vocabulary skill that correlates well with subject performance on independently developed human-written questions. In addition, strong correlations with standardized vocabulary tests point to the validity of our approach to automatic assessment of word knowledge.

Frontolimbic response to negative feedback in clinical depression

Functional neuroimaging suggests that limbic regions of the medial frontal cortex may be abnormally active in individuals with depression. These regions, including the anterior cingulate cortex, are engaged in both action regulation, such as monitoring errors and conflict, and affect regulation, such as responding to pain. The authors examined whether clinically depressed subjects would show abnormal sensitivity of frontolimbic networks as they evaluated negative feedback. Depressed subjects and matched control subjects performed a video game in the laboratory as a 256-channel EEG was recorded. Speed of performance on each trial was graded with a feedback signal of A, C, or F. By 350 ms after the feedback signal, depressed subjects showed a larger medial frontal negativity for all feedback compared with control subjects with a particularly striking response to the F grade. This response was strongest for moderately depressed subjects and was attenuated for subjects who were more severely depressed. Localization analyses suggested that negative feedback engaged sources in the anterior cingulate and insular cortices. These results suggest that moderate depression may sensitize limbic networks to respond strongly to aversive events.

Development of NeuroElectroMagnetic ontologies(NEMO): a framework for mining brainwave ontologies

Event-related potentials (ERP) are brain electrophysiological patterns created by averaging electroencephalographic (EEG) data, time-locking to events of interest (e.g., stimulus or response onset). In this paper, we propose a generic framework for mining anddeveloping domain ontologies and apply it to mine brainwave (ERP) ontologies. The concepts and relationships in ERP ontologies can be mined according to the following steps: pattern decomposition, extraction of summary metrics for concept candidates, hierarchical clustering of patterns for classes and class taxonomies, and clustering-based classification and association rules mining for relationships (axioms) of concepts. We have applied this process to several dense-array (128-channel) ERP datasets. Results suggest good correspondence between mined concepts and rules, on the one hand, and patterns and rules that were independently formulated by domain experts, on the other. Data mining results also suggest ways in which expert-defined rules might be refined to improve ontologyrepresentation and classification results. The next goal of our ERP ontology mining framework is to address some long-standing challenges in conducting large-scale comparison and integration of results across ERP paradigms and laboratories. In a more general context, this work illustrates the promise of an interdisciplinary research program, which combines data mining, neuroinformatics andontology engineering to address real-world problems.

Measuring incremental changes in word knowledge: Experimental validation and implications for learning and assessment

The goal of this study was to test a new technique for assessing vocabulary development. This technique is based on an algorithm for scoring the accuracy of word definitions using a continuous scale (Collins-Thompson & Callan, 2007). In an experiment with adult learners, target words were presented in six different sentence contexts, and the number of informative versus misleading contexts was systematically manipulated. Participants generated a target definition after each sentence, and the definition-scoring algorithm was used to assess the degree of accuracy on each trial. We observed incremental improvements in definition accuracy across trials. Moreover, learning curves were sensitive to the proportion of misleading contexts, the use of spaced versus massed practice, and individual differences, demonstrating the utility of this procedure for capturing specific experimental effects on the trajectory of word learning. We discuss the implications of these results for measurement of meaning, vocabulary assessment, and instructional design.

Predicting Robust Vocabulary Growth from Measures of Incremental Learning.

We report a study of incremental learning of new word meanings over multiple episodes. A new method called MESA (Markov Estimation of Semantic Association) tracked this learning through the automated assessment of learner-generated definitions. The multiple word learning episodes varied in the strength of contextual constraint provided by sentences, in the consistency of this constraint, and in the spacing of sentences provided for each trained word. Effects of reading skill were also examined. Results showed that MESA scores increased with each word learning encounter. MESA growth curves were affected by context constraint, spacing of practice, and reading skill. Most important, the accuracy of participant responses (MESA scores) during learning predicted which words would be retained over a 1-week period. These results support the idea that word learning is incremental and that partial gains in knowledge depend on properties of both the context and the learner. The introduction of MESA presents new opportunities to test word-learning theories and the complex factors that affect growth of word knowledge over time and in different contexts.

Automated protocol for evaluation of electromagnetic component separation (APECS): Application of a framework for evaluating statistical methods of blink extraction from multichannel EEG

OBJECTIVE We present APECS (Automated Protocol for Evaluation of Electromagnetic Component Separation), a framework for evaluating the accuracy of blind source separation algorithms in removing artifacts from EEG data. APECS applies multiple, automated procedures to quantify the extent to which blinks are removed, and the degree to which nonocular activity is left intact. METHODS APECS was used to evaluate blink removal using three BSS algorithms: Second-Order Blind Inference (SOBI) and two Independent Component Analysis (ICA) implementations, FastICA and Infomax. The algorithms were applied to a series of blink-free EEG datasets, which were contaminated with real or simulated blinks. Extracted components were assumed to contain blink activity if correlation of their spatial projectors to a predefined blink template exceeded some threshold, and if polarity inverted above and below the eyes. Blink-related components were then subtracted to produce filtered data. The success of each data decomposition is evaluated through the use of multiple, automated metrics, to determine which decomposition best approximates the ideal solution (complete separation of blink from nonblink activity). RESULTS The outcomes for the evaluation measures were generally congruent, but also provided different and complementary information about the quality of each data decomposition. Under our testing framework, Infomax outperformed both FastICA and SOBI. Best results were achieved when blink activity loaded onto a single component. CONCLUSIONS Multiple metrics, both quantitative and qualitative, are important in evaluating algorithms for artifact extraction. SIGNIFICANCE Failure to achieve complete separation of blink from nonblink activity can affect experimental outcomes, as illustrated here, using an ERP study of word-nonword discrimination. This illustrates the importance of methods for evaluation of artifact extraction results.

A framework to support automated classification and labeling of brain electromagnetic patterns

This paper describes a framework for automated classification and labeling of patterns in electroencephalographic (EEG) and magnetoencephalographic (MEG) data. We describe recent progress on four goals: 1) specification of rules and concepts that capture expert knowledge of event-related potentials (ERP) patterns in visual word recognition; 2) implementation of rules in an automated data processing and labeling stream; 3) data mining techniques that lead to refinement of rules; and 4) iterative steps towards system evaluation and optimization. This process combines top-down, or knowledge-driven, methods with bottom-up, or data-driven, methods. As illustrated here, these methods are complementary and can lead to development of tools for pattern classification and labeling that are robust and conceptually transparent to researchers. The present application focuses on patterns in averaged EEG (ERP) data. We also describe efforts to extend our methods to represent patterns in MEG data, as well as EM patterns in source (anatomical) space. The broader aim of this work is to design an ontology-based system to support cross-laboratory, cross-paradigm, and cross-modal integration of brain functional data. Tools developed for this project are implemented in MATLAB and are freely available on request.

CHAPTER 16 – The Neural Bases of Text and Discourse Processing

Understanding discourse requires the comprehension of individual words and sentences, as well as integration across sentence representations to form a coherent understanding of the discourse as a whole. The processes that achieve this coherence involve a dynamic interplay between mental rep resentations built on the current sentence, the prior discourse context, and the comprehender ’ s background (world) knowledge. In this chapter, we outline the cognitive and linguistic processes that support discourse comprehension and explore the functional neuroanatomy of text and discourse processing. Our review suggests an emerging picture of the neurocognition of discourse comprehension that involves an extended language processing network, including left dorsal and ventral frontal regions, left temporal cortex, medial frontal cortex, and posterior cingulate. While convergent evidence points to the importance of left frontal and temporal networks in discourse processing, the role of right hemisphere networks is less clear.

Ontology Database: A New Method for Semantic Modeling and an Application to Brainwave Data

We propose an automatic method for modeling a relational database that uses SQL triggers and foreign-keys to efficiently answer positive semantic queries about ground instances for a Semantic Web ontology. In contrast with existing knowledge-based approaches, we expend additional space in the database to reduce reasoning at query time. This implementation significantly improves query response time by allowing the system to disregard integrity constraints and other kinds of inferences at run-time. The surprising result of our approach is that load-time appears unaffected, even for medium-sized ontologies. We applied our methodology to the study of brain electroencephalographic (EEG and ERP) data. This case study demonstrates how our methodology can be used to proactively drive the design, storage and exchange of knowledge based on EEG/ERP ontologies.

Minimal Information for Neural Electromagnetic Ontologies (MINEMO): A standards-compliant method for analysis and integration of event-related potentials (ERP) data

We present MINEMO (Minimal Information for Neural ElectroMagnetic Ontologies), a checklist for the description of event-related potentials (ERP) studies. MINEMO extends MINI (Minimal Information for Neuroscience Investigations)to the ERP domain. Checklist terms are explicated in NEMO, a formal ontology that is designed to support ERP data sharing and integration. MINEMO is also linked to an ERP database and web application (the NEMO portal). Users upload their data and enter MINEMO information through the portal. The database then stores these entries in RDF (Resource Description Framework), along with summary metrics, i.e., spatial and temporal metadata. Together these spatial, temporal, and functional metadata provide a complete description of ERP data and the context in which these data were acquired. The RDF files then serve as inputs to ontology-based labeling and meta-analysis. Our ultimate goal is to represent ERPs using a rich semantic structure, so results can be queried at multiple levels, to stimulate novel hypotheses and to promote a high-level, integrative account of ERP results across diverse study methods and paradigms.