George Siemens

Learning analytics and educational data mining: towards communication and collaboration

Growing interest in data and analytics in education, teaching, and learning raises the priority for increased, high-quality research into the models, methods, technologies, and impact of analytics. Two research communities -- Educational Data Mining (EDM) and Learning Analytics and Knowledge (LAK) have developed separately to address this need. This paper argues for increased and formal communication and collaboration between these communities in order to share research, methods, and tools for data mining and analysis in the service of developing both LAK and EDM fields.

The Beginning of a Beautiful Friendship? Intelligent Tutoring Systems and MOOCs

A key challenge in ITS research and development is to support tutoring at scale, for example by embedding tutors in MOOCs. An obstacle to at-scale deployment is that ITS architectures tend to be complex, not easily deployed in browsers without significant server-side processing, and not easily embedded in a learning management system (LMS). We present a case study in which a widely used ITS authoring tool suite, CTAT/TutorShop, was modified so that tutors can be embedded in MOOCs. Specifically, the inner loop (the example-tracing tutor engine) was moved to the client by reimplementing it in JavaScript, and the tutors were made compatible with the LTI e-learning standard. The feasibility of this general approach to ITS/MOOC integration was demonstrated with simple tutors in an edX MOOC “Data Analytics and Learning.”

Learning Analytics The Emergence of a Discipline

Recently, learning analytics (LA) has drawn the attention of academics, researchers, and administrators. This interest is motivated by the need to better understand teaching, learning, “intelligent content,” and personalization and adaptation. While still in the early stages of research and implementation, several organizations (Society for Learning Analytics Research and the International Educational Data Mining Society) have formed to foster a research community around the role of data analytics in education. This article considers the research fields that have contributed technologies and methodologies to the development of learning analytics, analytics models, the importance of increasing analytics capabilities in organizations, and models for deploying analytics in educational settings. The challenges facing LA as a field are also reviewed, particularly regarding the need to increase the scope of data capture so that the complexity of the learning process can be more accurately reflected in analysis. Privacy and data ownership will become increasingly important for all participants in analytics projects. The current legal system is immature in relation to privacy and ethics concerns in analytics. The article concludes by arguing that LA has sufficiently developed, through conferences, journals, summer institutes, and research labs, to be considered an emerging research field.

Current state and future trends: a citation network analysis of the learning analytics field

This paper provides an evaluation of the current state of the field of learning analytics through analysis of articles and citations occurring in the LAK conferences and identified special issue journals. The emerging field of learning analytics is at the intersection of numerous academic disciplines, and therefore draws on a diversity of methodologies, theories and underpinning scientific assumptions. Through citation analysis and structured mapping we aimed to identify the emergence of trends and disciplinary hierarchies that are influencing the development of the field to date. The results suggest that there is some fragmentation in the major disciplines (computer science and education) regarding conference and journal representation. The analyses also indicate that the commonly cited papers are of a more conceptual nature than empirical research reflecting the need for authors to define the learning analytics space. An evaluation of the current state of learning analytics provides numerous benefits for the development of the field, such as a guide for under-represented areas of research and to identify the disciplines that may require more strategic and targeted support and funding opportunities.

Higher Education and the Promises and Perils of Social Networks

The last decade has produced tremendous innovation in how people connect with one another online. Social networks have experienced a rapid increase in popularity, producing both concerns (privacy, content ownership) and opportunities. The articles in this journal can be viewed as attempts to answer the question: What should educators do about social networks?

Penetrating the fog: analytics in learning and education

In the era of Internet, mobile technologies and open education, the need for changes to improve the efficiency and quality of higher education has become crucial.Big data and analytics can contribute to these changes and reshape the future of higher education. Basing decisions on data and evidence seems stunningly obvious. However, higher education, a field that gathers an astonishing array of data about its “customers,” has traditionally been inefficient in its data use, often operating with substantial delays in analyzing readily evident data and feedback. In this paper, the value of Analytics for Higher Education is discussed, and a model of learning analytics development is presented. The main pedagogical and ethical issues about the use of learning analytics are also pointed out, since learning is messy, and using analytics to describe learning is not easy. Nevertheless, Learning Analytics can penetrate the fog of uncertainty around the future of higher education, and shed light on how to allocate resources, develop competitive advantages, and most important, improve the quality and value of the learning experience.

Shared Task on Prediction of Dropout Over Time in Massively Open Online Courses

The shared task on Prediction of Dropout Over Time in MOOCs involves analysis of data from 6 MOOCs offered through Coursera. Data from one MOOC with approximately 30K students was distributed as training data and consisted of discussion forum data (in SQL) and clickstream data (in JSON format). The prediction task was Predicting Attrition Over Time. Based on behavioral data from a week’s worth of activity in a MOOC for a student, predict whether the student will cease to actively participate after that week. This paper describes the task. A full write up of the results is published separately (Rose & Siemens, 2014).

Towards automated content analysis of discussion transcripts: a cognitive presence case

In this paper, we present the results of an exploratory study that examined the problem of automating content analysis of student online discussion transcripts. We looked at the problem of coding discussion transcripts for the levels of cognitive presence, one of the three main constructs in the Community of Inquiry (CoI) model of distance education. Using Coh-Metrix and LIWC features, together with a set of custom features developed to capture discussion context, we developed a random forest classification system that achieved 70.3% classification accuracy and 0.63 Cohens kappa, which is significantly higher than values reported in the previous studies. Besides improvement in classification accuracy, the developed system is also less sensitive to overfitting as it uses only 205 classification features, which is around 100 times less features than in similar systems based on bag-of-words features. We also provide an overview of the classification features most indicative of the different phases of cognitive presence that gives an additional insights into the nature of cognitive presence learning cycle. Overall, our results show great potential of the proposed approach, with an added benefit of providing further characterization of the cognitive presence coding scheme.

Studying MOOC completion at scale using the MOOC replication framework

Research on learner behaviors and course completion within Massive Open Online Courses (MOOCs) has been mostly confined to single courses, making the findings difficult to generalize across different data sets and to assess which contexts and types of courses these findings apply to. This paper reports on the development of the MOOC Replication Framework (MORF), a framework that facilitates the replication of previously published findings across multiple data sets and the seamless integration of new findings as new research is conducted or new hypotheses are generated. In the proof of concept presented here, we use MORF to attempt to replicate 15 previously published findings across 29 iterations of 17 MOOCs. The findings indicate that 12 of the 15 findings replicated significantly across the data sets, and that two findings replicated significantly in the opposite direction. MORF enables larger-scale analysis of MOOC research questions than previously feasible, and enables researchers around the world to conduct analyses on huge multi-MOOC data sets without having to negotiate access to data.

Learning analytics and machine learning

Learning analytics (LA) as a field remains in its infancy. Many of the techniques now prominent from practitioners have been drawn from various fields, including HCI, statistics, computer science, and learning sciences. In order for LA to grow and advance as a discipline, two significant challenges must be met: 1) development of analytics methods and techniques that are native to the LA discipline, and 2) practitioners in LA to develop algorithms and models that reflect the social and computational dimensions of analytics. This workshop introduces researchers in learning analytics to machine learning (ML) and the opportunities that ML can provide in building next generation analysis models.