Dan Davis

Retrieval Practice and Study Planning in MOOCs: Exploring Classroom-Based Self-regulated Learning Strategies at Scale

Massive Open Online Courses (MOOCs) are successful in delivering educational resources to the masses, however, the current retention rates—well below 10 %—indicate that they fall short in helping their audience become effective MOOC learners. In this paper, we report two MOOC studies we conducted in order to test the effectiveness of pedagogical strategies found to be beneficial in the traditional classroom setting: retrieval practice (i.e. strengthening course knowledge through actively recalling information) and study planning (elaborating on weekly study plans). In contrast to the classroom-based results, we do not confirm our hypothesis, that small changes to the standard MOOC design can teach MOOC learners valuable self-regulated learning strategies.

Beyond the MOOC platform: gaining insights about learners from the social web

Massive Open Online Courses (MOOCs) have enabled millions of learners across the globe to increase their levels of expertise in a wide variety of subjects. Research efforts surrounding MOOCs are typically focused on improving the learning experience, as the current retention rates (less than 7% of registered learners complete a MOOC) show a large gap between vision and reality in MOOC learning. Current data-driven approaches to MOOC adaptations rely on data traces learners generate within a MOOC platform such as edX or Coursera. As a MOOC typically lasts between five and eight weeks and with many MOOC learners being rather passive consumers of the learning material, this exclusive use of MOOC platform data traces limits the insights that can be gained from them. The Social Web potentially offers a rich source of data to supplement the MOOC platform data traces, as many learners are also likely to be active on one or more Social Web platforms. In this work, we present a first exploratory analysis of the Social Web platforms MOOC learners are active on --- we consider more than 320,000 learners that registered for 18 MOOCs on the edX platform and explore their user profiles and activities on StackExchange, GitHub, Twitter and LinkedIn.

Open learner models and learning analytics dashboards: a systematic review

This paper aims to link student facing Learning Analytics Dashboards (LADs) to the corpus of research on Open Learner Models (OLMs), as both have similar goals. We conducted a systematic review of literature on OLMs and compared the results with a previously conducted review of LADs for learners in terms of (i) data use and modelling, (ii) key publication venues, (iii) authors and articles, (iv) key themes, and (v) system evaluation. We highlight the similarities and differences between the research on LADs and OLMs. Our key contribution is a bridge between these two areas as a foundation for building upon the strengths of each. We report the following key results from the review: in reports of new OLMs, almost 60% are based on a single type of data; 33% use behavioral metrics; 39% support input from the user; 37% have complex models; and just 6% involve multiple applications. Key associated themes include intelligent tutoring systems, learning analytics, and self-regulated learning. Notably, compared with LADs, OLM research is more likely to be interactive (81% of papers compared with 31% for LADs), report evaluations (76% versus 59%), use assessment data (100% versus 37%), provide a comparison standard for students (52% versus 38%), but less likely to use behavioral metrics, or resource use data (33% against 75% for LADs). In OLM work, there was a heightened focus on learner control and access to their own data.

Can Learners be Earners? Investigating a Design to Enable MOOC Learners to Apply their Skills and Earn Money in an Online Market Place

Massive Open Online Courses (MOOCs) aim to educate the world. More often than not, however, MOOCs fall short of this goal—a majority of learners are already highly educated (with a Bachelor’s degree or more) and come from specific parts of the (developed) world. Learners from developing countries without a higher degree are underrepresented, though desired, in MOOCs. One reason for those learners to drop out of a course can be found in their financial realities and the subsequent limited amount of time they can dedicate to a course besides earning a living. If we could pay learners to take a MOOC, this hurdle would largely disappear. With MOOCS, this leads to the following fundamental challenge: How can learners be paid at scale? Ultimately, we envision a recommendation engine that recommends tasks from online market places such as Upwork or witmart to learners, that are relevant to the course content of the MOOC. In this manner, the learners learn and earn money. To investigate the feasibility of this vision, in this paper, we explored to what extent (1) online market places contain tasks relevant to a specific MOOC, and (2) learners are able to solve real-world tasks correctly and with sufficient quality. Finally, based on our experimental design, we were also able to investigate the impact of real-world bonus tasks in a MOOC on the general learner population.

Toward large-scale learning design: categorizing course designs in service of supporting learning outcomes

This paper applies theory and methodology from the learning design literature to large-scale learning environments through quantitative modeling of the structure and design of Massive Open Online Courses. For two institutions of higher education, we automate the task of encoding pedagogy and learning design principles for 177 courses (which accounted for for nearly 4 million enrollments). Course materials from these MOOCs are parsed and abstracted into sequences of components, such as videos and problems. Our key contributions are (i) describing the parsing and abstraction of courses for quantitative analyses, (ii) the automated categorization of similar course designs, and (iii) the identification of key structural components that show relationships between categories and learning design principles. We employ two methods to categorize similar course designs---one aimed at clustering courses using transition probabilities and another using trajectory mining. We then proceed with an exploratory analysis of relationships between our categorization and learning outcomes.

Evaluating retrieval practice in a MOOC: how writing and reading summaries of videos affects student learning

Videos are often the core content in open online education, such as in Massive Open Online Courses (MOOCs). Students spend most of their time in a MOOC on watching educational videos. However, merely watching a video is a relatively passive learning activity. To increase the educational benefits of online videos, students could benefit from more actively interacting with the to-be-learned material. In this paper two studies (n = 13k) are presented which examined the educational benefits of two more active learning strategies: 1) Retrieval Practice tasks which asked students to shortly summarize the content of videos, and 2) Given Summary tasks in which the students were asked to read pre-written summaries of videos. Writing, as well as reading summaries of videos were positively related to quiz grades. Both interventions seemed to help students to perform better, but there was no apparent difference between the efficacy of these interventions. These studies show how the quality of online education can be improved by adapting course design to established approaches from the learning sciences.

SRLx: A Personalized Learner Interface for MOOCs

Past research in large-scale learning environments has found one of the most inhibiting factors to learners’ success to be their inability to effectively self-regulate their learning efforts. In traditional small-scale learning environments, personalized feedback (on progress, content, behavior, etc.) has been found to be an effective solution to this issue, but it has not yet widely been evaluated at scale. In this paper we present the Personalized SRL Support System (SRLx), an interactive widget that we designed and open-sourced to improve learners’ self-regulated learning behavior in the Massive Open Online Course platform edX. SRLx enables learners to plan their learning on a weekly basis and view real-time feedback on the realization of those plans. We deployed SRLx in a renewable energies MOOC to more than 2,900 active learners and performed an exploratory analysis on our learners’ SRL behavior.

Buying time: enabling learners to become earners with a real-world paid task recommender system

Massive Open Online Courses (MOOCs) aim to educate the world, especially learners from developing countries. While MOOCs are certainly available to the masses, they are not yet fully accessible. Although all course content is just clicks away, deeply engaging with a MOOC requires a substantial time commitment, which frequently becomes a barrier to success. To mitigate the time required to learn from a MOOC, we here introduce a design that enables learners to earn money by applying what they learn in the course to real-world marketplace tasks. We present a Paid Task Recommender System (Rec-

Workshop on integrated learning analytics of MOOC post-course development

ys), which automatically recommends course-relevant tasks to learners as drawn from online freelance platforms. Rec-

Follow the successful crowd: raising MOOC completion rates through social comparison at scale

ys has been deployed into a data analysis MOOC and is currently under evaluation.