Sarah Giersch

A study of teachers' use of online learning resources to design classroom activities

While much progress has been made on the technical design and development of digital libraries, much less is known about how and why education digital library content and associated tools can support and enhance the activities of educators in their professional work. This article elaborates a conceptual framework that characterizes teachers’ practices when using online learning resources (called ‘teaching as design’), and a professional development model aimed at increasing teachers’ capacity for designing learning activities in the context of authentic practice. Findings from two workshop implementations showed positive impacts on teachers’ knowledge, attitudes, and subsequent behaviours using online learning resources. An analysis of teacher created activities indicates a relationship between the form of design (offload, adaptation, or improvisation) and the granularity of the learning objects utilized in the activity.

Creating virtual collections in digital libraries: benefits and implementation issues

Digital libraries have the potential to not only duplicate many of the services provided by traditional libraries but to extend them. Basic finding aids such as search and browse are common in most of todays digital libraries. But just as a traditional library provides more than a card catalog and browseable shelves of books, an effective digital library should offer a wider range of services. Using the traditional library concept of special collections as a model, in this paper we propose that explicitly defining sub-collections in the digital library-virtual collections-can benefit both the librarys users and contributors and increase its viability. We first introduce the concept of a virtual collection, outline the costs and benefits for defining such collections, and describe an implementation of collection-level metadata to create virtual collections for two different digital libraries. We conclude by discussing the implications of virtual collections for enhancing interoperability and sharing across digital libraries, such as those that are part of the National SMETE Digital Library.

Developing recommendation services for a digital library with uncertain and changing data

In developing recommendation services for a new digital library called iLumina (www.ilumina-project.org), we are faced with several challenges related to the nature of the data we have available. The availability and consistency of data associated with iLumina is likely to be highly variable. Any recommendation strategy we develop must be able to cope with this fact, while also being robust enough to adapt to additional types of data available over time as the digital library develops. In this paper we describe the challenges we are faced with in developing a system that can provide our users with good, consistent recommendations under changing and uncertain conditions.

Developing a review process for online resources

The democratization of content creation via ubiquitous Internet tools and infrastructure [1] has fueled an explosion of user-generated content in the commercial and educational markets. Indeed, funding agencies such as the National Science Foundation (NSF) are actively seeking ways to integrate teachers and learners into the education cyber-infrastructure, whereby they become co-creators of educational content [2]. The ease with which this content, often in the form of online learning resources of varying levels of granularity, can be created and disseminated places it outside the usual peer review processes employed by publishers and professional societies. To date, digital library (DL) developers, teachers and school administrators, concerned whether teachers are using peerreviewed online learning resources, have depended on one or a combination of the following proxies to establish an imprimatur of quality: the reputation and oversight of a funding organization (e.g., NSFs peer review process), the credentials of the content creator (e.g., National Science Teachers Association) or the collection development policies of specific DLs (e.g., DLESE). Now more than ever, though, sites such as YouTube, Flickr and ccMixter and the evolving education cyber-infrastructure, have created an environment where user-generated content is beyond the reach of even these proxy review processes. However, in the omnipresent climate of accountability within K12 education at U.S. federal, state and local levels, education DLs are being challenged to identify the value: of the resources they hold and services they provide to users; and, of what their users create with those resources. For all of these reasons, it is useful, and necessary, to develop a standardized rubric and process to review online education resources. In particular, this work should leverage social and technical networks to enrich, facilitate, and automate the review process. The Digital Libraries go to School project was funded by NSF in 2006 to develop a professional development workshop curriculum that enables teachers to use the Instructional Architect (IA; http://ia.usu.edu) to design their own learning activities for classrooms using online STEM resources from the National Science Digital Library (NSDL.org) and the wider Web. One component of the project is to examine the criteria and approaches for reviewing the quality of teacher-created online learning resources in order to develop a rubric and workflow process. Work to date includes conducting focus groups and surveys with teachers and a 5-person Expert Review Committee, complemented by a literature review to identify elements for a review rubric incorporating the work of other education DLs (e.g., DLESE, MERLOT, NEEDS, among others). Findings are being synthesized, and based on analysis, a draft list of elements has been identified for further testing in Spring 2008. At the same time, a workflow process for conducting reviews with teacher-created resources will be piloted. It will combine human-generated reviews with machine-generated information about online resources (e.g., image and word count; educational standards alignment; currency of updates, provenance) [3]. Further work will identify areas for improving the review rubric and scaling and standardizing the workflow process for Fall 2008. We will also evaluate the usefulness of the reviews to teachers, and to stakeholders such as the IA, NSDL, NSF and other DLs, in providing access to high-quality online content.

Qualitative Analysis of Association of Research Libraries' E‐Metrics Participant Feedback about the Evolution of Measures for Networked Electronic Resources

The Association of Research Libraries’ (ARL) E‐Metrics Project is an ongoing effort to develop new measures that describe and measure networked electronic resources and also to underscore the need for measuring the value of such resources. This article presents results from an ongoing iterative qualitative study with the following goals: (a) to understand ARL libraries’ needs for evaluating electronic resources; (b) to refine proposed definitions for describing and measuring electronic resources; (c) to institutionalize data collection for ongoing, annual reporting purposes; and (d) to develop reliable indicators for summative evaluation of electronic resources. Qualitative data collected from participants attending the initial meeting to develop the E‐Metrics Project in 2000 are analyzed and compared with an ongoing analysis of interview data collected from the 2003–4 E‐metrics participants. The evolving nature of the formats, methods of access, and acquisition plans for electronic resources present challenges in describing and measuring them. New methodologies that use a mixed‐methods approach should also evolve to support creative and innovative ways for evaluating an increasingly complex information environment.

Developing a review rubric for learning resources in digital libraries

This paper describes the development of a review rubric for learning resources in the context of the Instructional Architect (IA), a web-based authoring tool. We describe the motivation for developing a review rubric, the process for creating it by synthesizing the rubrics of other education-related digital libraries, and the results of testing the rubric with teachers. Analysis of usability and reliability indicates that the review rubric influences how teachers design online learning resources.

A tale of two studies: is dissemination working?

In this paper we describe preliminary results from two ongoing research projects that investigate the dissemination practices surrounding digital STEM learning materials for undergraduates. This research consists of two related studies, : 1) survey research about the dissemination practices of NSF-funded PIs; and, 2) a case study on the dissemination practices of courseware developers who won the Premier Award for Excellence in Engineering Education. The vast majority of PIs reported in the survey that they do not take advantage of digital dissemination methods such as education digital libraries. Premier Award-winning innovators reported using multiple dissemination methods - traditional and digital. Recommendations are provided regarding how digital library developers might work with PIs to improve dissemination.

Steps towards establishing shared evaluation goals and procedures in the National Science Digital Library

A community-based process was used to develop shared evaluation goals and instruments to begin evaluating the National Science Digital Library (NSDL). Results from a pilot study examining library usage, collections growth, and library governance processes are reported. The methods used in the pilot included Web log usage analysis, collections assessment techniques, survey instruments, and semistructured interviews.

Toward a sharable digital library of reusable teaching resources

COMMUNICATIONS OF THE ACM May 2001/Vol. 44, No. 5 79 iLumina is starting with resources from its partner institutions. These include materials from the Computer Science Teaching Center (CSTC; www.cstc.org), and the ACM SIGGRAPH Education Committee’s Digital Library (SECDL; www.education.siggraph.org), as well as new digital teaching materials from UNCW. Resources from these sites are not placed directly into iLumina, but reside in separate content repositories. iLumina itself actually contains only metadata associated with these resources. In short, iLumina is a partly distributed and partly centralized library. ILumina’s rich metadata, derived from the IMS specifications [1], not only enables powerful searches, but is also the basis for several secondtier user services, including:

Pilot testing the DigiQUAL TM protocol: lessons learned

The association of research libraries (ARL) is developing the DigiQUALtrade protocol to assess the service quality provided by digital libraries (DLs). In 2005, statements about DL service quality were put through a two-step validation process with DL developers and then with users in an online survey.