Mimi Recker

What Do You Recommend? Implementation and Analyses of Collaborative Information Filtering of Web Resources for Education.

This article examines results from one pilotstudy and two empirical studies of acollaborative filtering (CF) system applied ineducational settings. CF is a populartechnology in electronic commerce, whichleverages the interests of entire communitiesto provide targeted, personalizedrecommendations of interesting products orresources to individuals. In electroniccommerce, entertainment, and related domains,CF has proven an accurate and reliable tool;yet educational applications remain limited.From analyses of data from these three studies,we believe that CF holds promise in educationnot only for the purposes of helping learnersand educators find useful resources forlearning, but as a means of bringing togetherpeople with similar interests and beliefs, andpossibly as an aid to the learning processitself.

A Non-authoritative Educational Metadata Ontology for Filtering and Recommending Learning Objects

Digital libraries populated with learning objects are becoming popular tools in the creation of instructional technologies. Many current efforts to create standard metadata structures that facilitate the discovery and instructional use of learning objects recommend a single, authoritative metadata record per version of the learning object. However, as we argue in this paper, a single metadata record — particularly one with fields that emphasize knowledge management and technology, while evading instructional issues — provides information insufficient to support instructional utilization decisions. To put learning objects to instructional use, users must examine the individual objects, forfeiting the supposed benefits of the metadata system. As a solution, we propose a system that includes multi-record, non-authoritative metadata focussed on the surrounding instructional context of learning objects.

You can lead a horse to water: teacher development and use of digital library resources

This article presents findings from approximately 150 users who created instructional projects using educational digital library resources. One hundred of these users were teachers participating in professional development workshops on the topic of digital libraries. Our iterative approach to tool and workshop development and implementation was based on a framework that characterizes several input, output, and process variables affecting dissemination of such technologies in educational contexts. Data sources involved a mix of qualitative and quantitative methods, including electronic surveys, interviews, participant observations, and server log file and artifact analyses. These multiple and complementary levels of analyses reveal that despite teachers reporting great value in learning resources and educational digital libraries, significant and lasting impact on teaching practice remains difficult to obtain

A study of teachers' use of online learning resources to design classroom activities

While much progress has been made on the technical design and development of digital libraries, much less is known about how and why education digital library content and associated tools can support and enhance the activities of educators in their professional work. This article elaborates a conceptual framework that characterizes teachers’ practices when using online learning resources (called ‘teaching as design’), and a professional development model aimed at increasing teachers’ capacity for designing learning activities in the context of authentic practice. Findings from two workshop implementations showed positive impacts on teachers’ knowledge, attitudes, and subsequent behaviours using online learning resources. An analysis of teacher created activities indicates a relationship between the form of design (offload, adaptation, or improvisation) and the granularity of the learning objects utilized in the activity.

Integrating Technology and Problem-based Learning: A Mixed Methods Study of Two Teacher Professional Development Designs

This article describes two consecutive enactments of technology-oriented teacher professional development designs, aimed at helping teachers find high-quality online learning resources and use them in designing effective problem-based learning (PBL) activities for their students. To align with current professional development prescriptions, in the first enactment, teachers learned PBL design skills concurrently with technology skills. Following aspects of design-based research, the professional development theory, participant feedback, and results from the first enactment informed the design of the second. In this second enactment, technology skills were separated and presented prior to learning about PBL. Results from a mixed-methods study of impact indicated that both professional development enactments were associated with large increases in teacher knowledge, experience, and confidence with regards to technology use and integration. Variations in the level of PBL usage by teachers in their activities, and the degree to which they discuss PBL and technology integration are presented alongside limitations, practical significance, scholarly significance, and planned future work.

Teaching, Designing, and Sharing: A Context for Learning Objects

This article describes a professional development model and a set of tools intended to increase teachers’ capacity for the design of instructional activities using learning objects. It then reports preliminary findings from studies involving teachers (n=41) who participated in the professional development workshops based on the model. Findings suggest that participants saw many potential benefits of using online resources in support of teaching, including their convenience and currency. In terms of creating learning activities for their students, the most common use mentioned was for enrichment purposes. Analyses also showed that participants seemed to prefer to use small granularity resources. However, despite this enthusiasm, post-workshop usage was generally low. Participants also identified barriers in using online resources. These included lack of technology access and literacy, and difficulties in managing the overabundance of resources and their varying quality. Participants also mentioned the importance of accessing online resources for research purposes.

An evaluation model for a digital library services tool

This paper describes an evaluation model for a digital library tool, the Instructional Architect, which enables users to discover, select, reuse, sequence, and annotate digital library learning objects. By documenting our rapid-prototyping, iterative, and user-centered approach for evaluating a digital library service, we provide a model and set of methods that other developers may wish to employ. In addition, we provide preliminary results from our studies.

Developing a review process for online resources

The democratization of content creation via ubiquitous Internet tools and infrastructure [1] has fueled an explosion of user-generated content in the commercial and educational markets. Indeed, funding agencies such as the National Science Foundation (NSF) are actively seeking ways to integrate teachers and learners into the education cyber-infrastructure, whereby they become co-creators of educational content [2]. The ease with which this content, often in the form of online learning resources of varying levels of granularity, can be created and disseminated places it outside the usual peer review processes employed by publishers and professional societies. To date, digital library (DL) developers, teachers and school administrators, concerned whether teachers are using peerreviewed online learning resources, have depended on one or a combination of the following proxies to establish an imprimatur of quality: the reputation and oversight of a funding organization (e.g., NSFs peer review process), the credentials of the content creator (e.g., National Science Teachers Association) or the collection development policies of specific DLs (e.g., DLESE). Now more than ever, though, sites such as YouTube, Flickr and ccMixter and the evolving education cyber-infrastructure, have created an environment where user-generated content is beyond the reach of even these proxy review processes. However, in the omnipresent climate of accountability within K12 education at U.S. federal, state and local levels, education DLs are being challenged to identify the value: of the resources they hold and services they provide to users; and, of what their users create with those resources. For all of these reasons, it is useful, and necessary, to develop a standardized rubric and process to review online education resources. In particular, this work should leverage social and technical networks to enrich, facilitate, and automate the review process. The Digital Libraries go to School project was funded by NSF in 2006 to develop a professional development workshop curriculum that enables teachers to use the Instructional Architect (IA; http://ia.usu.edu) to design their own learning activities for classrooms using online STEM resources from the National Science Digital Library (NSDL.org) and the wider Web. One component of the project is to examine the criteria and approaches for reviewing the quality of teacher-created online learning resources in order to develop a rubric and workflow process. Work to date includes conducting focus groups and surveys with teachers and a 5-person Expert Review Committee, complemented by a literature review to identify elements for a review rubric incorporating the work of other education DLs (e.g., DLESE, MERLOT, NEEDS, among others). Findings are being synthesized, and based on analysis, a draft list of elements has been identified for further testing in Spring 2008. At the same time, a workflow process for conducting reviews with teacher-created resources will be piloted. It will combine human-generated reviews with machine-generated information about online resources (e.g., image and word count; educational standards alignment; currency of updates, provenance) [3]. Further work will identify areas for improving the review rubric and scaling and standardizing the workflow process for Fall 2008. We will also evaluate the usefulness of the reviews to teachers, and to stakeholders such as the IA, NSDL, NSF and other DLs, in providing access to high-quality online content.

Developing a review rubric for learning resources in digital libraries

This paper describes the development of a review rubric for learning resources in the context of the Instructional Architect (IA), a web-based authoring tool. We describe the motivation for developing a review rubric, the process for creating it by synthesizing the rubrics of other education-related digital libraries, and the results of testing the rubric with teachers. Analysis of usability and reliability indicates that the review rubric influences how teachers design online learning resources.

Automating open educational resources assessments: a machine learning generalization study

Assessing the quality of online educational resources in a cost effective manner is a critical issue for educational digital libraries. This study reports on the approach for extending the Open Educational Resource Assessments (OPERA) algorithm from assessing vetted to peer-produced content. This article reports details of changes to the algorithm, comparisons between human raters and the algorithm, and the extent the algorithm can automate the review process.