Sarah C. Harwell

Automatic metadata generation & evaluation

The poster reports on a project in which we are investigating methods for breaking the human metadata-generation bottleneck that plagues Digital Libraries. The research question is whether metadata elements and values can be automatically generated from the content of educational resources, and correctly assigned to mathematics and science educational materials. Natural Language Processing and Machine Learning techniques were implemented to automatically assign values of the GEMgenerate metadata element set tofor learning resources provided by the Gateway for Education (GEM), a service that offers web access to a wide range of educational materials. In a user study, education professionals evaluated the metadata assigned to learning resources by either automatic tagging or manual assignment. Results show minimal difference in the eyes of the evaluators between automatically generated metadata and manually assigned metadata.

Breaking the metadata generation bottleneck: preliminary findings

The goal of our 18 month NSDL-funded project is to develop Natural Language Processing and Machine Learning technology which will accomplish automatic metadata generation for individual educational resources in digital collections. The metadata tags that the system will be learning to automatically assign are the full complement of Gateway to Educational Materials (GEM) metadata tags – from the nationally recognized consortium of organizations concerned with access to educational resources. The documents that comprise the sample for this research come from the Eisenhower National Clearinghouse on Science and Mathematics.

Text Categorization for Aligning Educational Standards

Standard alignment (where standards describing similar concepts are correlated) is a necessary task in providing full access to educational resources. Manual alignment is time consuming and expensive. We propose an automatic alignment system, using machine learning techniques utilizing natural language processing. In this paper we discuss our experiments on text categorization for automatic alignment. We explore the role of relevant vocabulary sets in automatic alignment

Standards alignment for metadata assignment

The research in this paper describes a Machine Learning technique called hierarchical text categorization which is used to solve the problem of finding equivalents from among different state and national education standards. The approach is based on a set of manually aligned standards and utilizes the hierarchical structure present in the standards to achieve a more accurate result. Details of this approach and its evaluation are presented.

Modeling Reference Interviews as a Basis for Improving Automatic QA Systems

The automatic QA system described in this paper uses a reference interview model to allow the user to guide and contribute to the QA process. A set of system capabilities was designed and implemented that defines how the users contributions can help improve the system. These include tools, called the Query Template Builder and the Knowledge Base Builder, that tailor the document processing and QA system to a particular domain by allowing a Subject Matter Expert to contribute to the query representation and to the domain knowledge. During the QA process, the system can interact with the user to improve query terminology by using Spell Checking, Answer Type verification, Expansions and Acronym Clarifications. The system also has capabilities that depend upon, and expand the users history of interaction with the system, including a User Profile, Reference Resolution, and Question Similarity modules

Generating and evaluating automatic metadata for educational resources

Metadata provides a higher-level description of digital library resources and serves as a searchable record for browsing and accessing digital library content. However, manually assigning metadata is a resource-consuming task for which Natural Language Processing (NLP) can provide a solution. This poster coalesces the findings from research and development accomplished across two multi-year digital library metadata generation and evaluation projects and suggests how the lessons learned might benefit digital libraries with the need for high-quality, but efficient metadata assignment for their resources.