Candace Thille

The Open Learning Initiative: Measuring the Effectiveness of the OLI Statistics Course in Accelerating Student Learning.

The Open Learning Initiative (OLI) is an open educational resources project at Carnegie Mellon University that began in 2002 with a grant from The William and Flora Hewlett Foundation. OLI creates web-based courses that are designed so that students can learn effectively without an instructor. In addition, the courses are often used by instructors to support and complement face-to-face classroom instruction. Our evaluation efforts have investigated OLI courses’ effectiveness in both of these instructional modes – stand-alone and hybrid. This report documents several learning effectiveness studies that were focused on the OLI-Statistics course and conducted during Fall 2005, Spring 2006, and Spring 2007. During the Fall 2005 and Spring 2006 studies, we collected empirical data about the instructional effectiveness of the OLI-Statistics course in stand-alone mode, as compared to traditional instruction. In both of these studies, in-class exam scores showed no significant difference between students in the stand-alone OLI-Statistics course and students in the traditional instructor-led course. In contrast, during the Spring 2007 study, we explored an accelerated learning hypothesis, namely, that learners using the OLI course in hybrid mode will learn the same amount of material in a significantly shorter period of time with equal learning gains, as compared to students in traditional instruction. In this study, results showed that OLI-Statistics students learned a full semester’s worth of material in half as much time and performed as well or better than students learning from traditional instruction over a full semester. Editor: Stephen Godwin (Open University, UK). Reviewers: Tim de Jong (Open University, NL), Elia Tomadaki (Open University, UK), and Stephen Godwin (Open University, UK). Interactive elements: A demonstration of the StatTutor statistics tutorial is available for playback from http://jime.open.ac.uk/2008/14/stattutor_tour/ . The demonstration is in Flash format. http://jime.open.ac.uk/2008/14/stattutor_tour/

Open Learning Initiative

Launched at Carnege Mellon University in the fall of 2002, the Open Learning Initiative (OLI) is dedicated to the development of freely available, stand-alone college-level online courses informed by the best current research from the cognitive and learning sciences.1 The OLI course-design process is unique in its dedication to teaming faculty content experts with cognitive scientists, learning scientists, human-computer interaction specialists, formative assessment specialists, and programmers. The initiative’s commitment to ongoing course evaluation and iterative improvement also sets it apart. Ultimately, the collaborative nature of the OLI course-design process has had an additional, unanticipated effect: inspiring participating faculty members to rethink their approach to classroom teaching at the university. Although OLI courses are designed as stand-alone experiences, Carnegie Mellon faculty are successfully integrating OLI’s Web-based instruction modules into their traditional instructor-led courses.

Cold Rolled Steel and Knowledge: What Can Higher Education Learn About Productivity?

director of the Open Learning Initiative (OLI) at Carnegie Mellon University. She is the co-director of OLnet and serves as a redesign scholar for NCAT, as a fellow of International Society for Design and Development in Education, and on the Global Executive Advisory board for HP’s Catalyst Initiative. She worked with the US Dept. of Education on the 2010 National Education Technology Plan. Joel Smith is vice provost and chief information officer at Carnegie Mellon University. He also directs Carnegie Mellon’s Office of Technology for Education and is co-PI on the OLI.

MOOCs and Technology to Advance Learning and Learning Research Opening Statement: MOOCs and technology to advance learning and learning research (Ubiquity symposium)

MOOCs have fueled both hope and anxiety about the future of higher education. Our objective in this symposium is to surface and explore some of the open questions which have arisen in the MOOC debates. In this symposium, ten authors examine different aspects of MOOCs and technology to advance learning and learning research.

Community based educational data repositories and analysis tools

This workshop will explore community based repositories for educational data and analytic tools that are used to connect researchers and reduce the barriers to data sharing. Leading innovators in the field, as well as attendees, will identify and report on bottlenecks that remain toward our goal of a unified repository. We will discuss these as well as possible solutions. We will present LearnSphere, an NSF funded system that supports collaborating on and sharing a wide variety of educational data, learning analytics methods, and visualizations while maintaining confidentiality. We will then have hands-on sessions in which attendees have the opportunity to apply existing learning analytics workflows to their choice of educational datasets in the repository (using a simple drag-and-drop interface), add their own learning analytics workflows (requires very basic coding experience), or both. Leaders and attendees will then jointly discuss the unique benefits as well as the limitations of these solutions. Our goal is to create building blocks to allow researchers to integrate their data and analysis methods with others, in order to advance the future of learning science.

OARS: exploring instructor analytics for online learning

Learning analytics systems have the potential to bring enormous value to online education. Unfortunately, many instructors and platforms do not adequately leverage learning analytics in their courses today. In this paper, we report on the value of these systems from the perspective of course instructors. We study these ideas through OARS, a modular and real-time learning analytics system that we deployed across more than ten online courses with tens of thousands of learners. We leverage this system as a starting point for semi-structured interviews with a diverse set of instructors. Our study suggests new design goals for learning analytics systems, the importance of real-time analytics to many instructors, and the value of flexibility in data selection and aggregation for an instructor when working with an analytics system.

Exploring the impact of the default option on student engagement and performance in a statistics MOOC

Engagement and motivation are particularly important in optional learning environments, like educational games and massive open online courses. Providing some aspects of autonomy and choice to the student can yield significant benefits to learner motivation and persistence; yet there is also evidence that unsupported learners may not always automatically choose to allocate their learning time to pedagogical activities that are most known to be as associated with better learning outcomes. We investigated the impact of choice on student engagement and learning in a Massive Open Online Course (MOOC) on introductory statistics and probability. We compared conditions in which students are given free choice over the practice problems completed to conditions in which students receive a full set of practice activities or no practice activities before completing a post-test. In all cases students were free to navigate to other sections of the course at any time. In one of the two topic sections that included personalized practice activities we found that students performed better in the condition in which they were prompted to complete all practice activities. Though more students in this condition dropped out before reaching the post-test, many more students completed the full set of practice activities in this section than those who did in the free choice condition. These results are still quite preliminary but suggest that providing a default encouraged opt in procedure can encourage students to do more problems than they would otherwise, and that doing such additional problems can yield learning gains.

Estimating the minimum number of opportunities needed for all students to achieve predicted mastery

We have conducted a study on how many opportunities are necessary, on average, for learners to achieve mastery of a skill, also called a knowledge component (KC), as defined in the Open Learning Initiative (OLI) digital courseware. The study used datasets from 74 different course instances in four topic areas comprising 3813 students and 1.2 million transactions. The analysis supports our claim that the number of opportunities to reach mastery gives us new information on both students and the development of course components. Among the conclusions are a minimum of seven opportunities are necessary for each knowledge component, more if the prior knowledge among students are uneven within a course. The number of KCs in a course increases the number of opportunities needed. The number of opportunities to reach mastery can be used to identify KCs that are outliers that may be in need of better explanations or further instruction.