David Boulanger

Smart Learning Analytics

A smart learning environment (SLE) is characterized by the key provision of personalized learning experiences. To approach different degrees of personalization in online learning, this paper introduces a framework called SCALE that tracks finer level learning experiences and translates them into opportunities for custom feedback. A prototype version of the SCALE system has been used in a study to track the habits of novice programmers. Growth of coding competencies of first year engineering students has been captured in a continuous manner. Students have been provided with customized feedback to optimize their learning path in programming. This paper describes key aspects of our research with the SCALE system and highlights results of the study.

Unfolding Learning Analytics for Big Data

Educational applications, in general, treat disparate study threads as a singular entity, bundle pedagogical intervention and other student support services at a coarser level, and summatively assess final products of assessments. In this research, we propose an analytics framework where we closely monitor individual threads of study habits and assess study threads in an individual fashion to trace learning processes leading into assessment products. We developed customized intervention to target specific skills and nurture optimal study habits. The framework has been implemented in a system called SCALE (Smart Causal Analytics on LEarning). SCALE enables the tracking of students’ individual study threads towards multiple final study products. The large volume, multiple variety, and incessant flow of data classifies our work in the realms of big data analytics. We conducted a preliminary study using SCALE. The results show the ability of the system to track the evolution of competencies. We propose that explicitly supporting the development of a targeted set of competencies is one of the key tenets of Smart Learning Environments.

Big Data Learning Analytics: A New Perpsective

Learners’ attainment of academic knowledge in postsecondary institutions is predominantly expressed by summative approaches. Instead, recent advances in educational technology have hinted at a means to continuously measure learning attainment in terms of personalized learner competency, capacity, and effectiveness. Similarly, educational technology also offers guidelines to continuously measure instructional attainment in terms of instructional competency, instructional capacity, and instructional effectiveness. While accurate computational models that embody these attainments, educational and instructional, remain a distant and elusive goal, big data learning analytics approaches this goal by continuously observing study experiences and instructional practices at various levels of granularity, and by continually constructing and using models from these observations. This article offers a new perspective on learning and instructional attainments with big data analytics as the underlying framework, discusses approaches to this framework with evidences from the literature, and offers a case study that illustrates the need to pursue research directions arising from this new perspective.

SCALE: A Competence Analytics Framework

This paper introduces SCALE, a Smart Competence Analytics engine on LEarning, as a framework to implement content analysis in several learning domains and provide mechanisms to define proficiency and confidence metrics. SCALE’s ontological design plays a crucial role in centralizing and homogenizing disparate data from domain-specific parsers and ultimately from several learning domains. This paper shows how SCALE has been applied in the programming domain and reveals systematically how the work content of a student can be analyzed and converted to evidences to assess his/her proficiency in domain-specific competences and how SCALE can also analyze the student’s interaction with a learning activity and provide a confidence metric to assess his/her behavior as he/she culminates toward goal achievements.

Performance Analysis of Parallel Particle Swarm Optimization Based Clustering of Students

While accurate computational models that embody learning efficiency remain a distant and elusive goal, big data learning analytics approaches this goal by recognizing competency growth of learners, at various levels of granularity, using a combination of continuous, formative, and summative assessments. Our earlier research employed the conventional Particle Swarm Optimization (PSO) based clustering mechanism to cluster large numbers of learners based on their observed study habits and the consequent growth of subject knowledge competencies. This paper describes a Parallel Particle Swarm Optimization (PPSO) based clustering mechanism to cluster learners. Using a simulation study, performance measures of quality of clusters such as the Inter Cluster Distance, the Intra Cluster Distance, the processing time and the acceleration values are estimated and compared.

Learning Analytics in the Energy Industry: Measuring Competences in Emergency Procedures

Several major accidents in the oil and gas industry traced their source to deficient training resulting in serious injuries and even casualties along with extremely expensive damage to equipment and decrease in productivity. This paper presents a procedure evaluation/e-training tool called PeT to track the knowledge and confidence of trainees in emergency operating procedures. PeT was tested with two emergency procedures in an oil and gas company in Canada. A text-based knowledge test was implemented for each procedure. Each test consisted of multiple-choice questions. Answers were classified as perfectly correct, incomplete but correct, partially correct, mostly incorrect, and totally incorrect. The paper also describes the six-factor confidence model underlying the confidence computations in PeT: knowledge, reaction time, lingering, number of visits (revision), number of selections, and number of switching answers. Each confidence factor measures a specific aspect of the targeted behaviour in an emergency. The results of two experiments conducted in 2014 in an oil and gas company are also presented to show the types of analysis that PeT enables. A plan to move PeT into an interactive training environment to track the actions of operators in their work environment and translate their interaction into higher level competences is also briefly introduced.

Real-Time Visual Feedback: A Study in Coding Analytics

Higher dropout and failure rates among computer science students in introductory programming courses tend to be a norm for many institutions. Years of evidence indicate that dropouts and failures persist in spite of advancements in pedagogy, technology, and teacher training. Most advancements have relied on summative assessments and of late formative assessments. This research explores assessments computed from real-time measures, based on observational data collected during student engagement with study and remedial activities. An experiment was conducted to measure the impact of real-time code assessment and dashboard-based feedback in the domain of Programming. Results indicate better course grades for a small percentage of students, and the need for task-level and meta-level interactions to guarantee significant and persistent academic performance and programming mastery.

Measurement of Quality of a Course

Traditionally, the quality of a course offering is measured based on learner feedback at the end of the offering. This chapter offers a method to measure the quality of a course offering—continually, formatively, and summatively—using factors such as the quality of resources used, learner motivation, learner capacity, learner competency growth, and instructor competence. These factors are represented in a Bayesian belief network (BBN) in a system called MI-IDEM. MI-IDEM receives streams of data corresponding to these factors as and when they become available, which leads to estimates of quality of the course offering based on individual factors as well as an overall quality of the offering. Continuous, formative, and summative course quality measurements are imperative to identify weaknesses in the learning process of students and to assist them when they need help. This chapter professes the need for a comprehensive measurement of course quality and ensuing initiatives to personalize and adapt course offerings. It presents two case studies of this novel approach: first, measurement of the quality of a course offering in a blended online learning environment and second, measurement of the quality of training course offering in an industry environment.

Deep Learning in Automated Essay Scoring.

This paper explores the application of deep learning in automated essay scoring (AES). It uses the essay dataset #8 from the Automated Student Assessment Prize competition, hosted by the Kaggle platform, and a state-of-the-art Suite of Automatic Linguistic Analysis Tools (SALAT) to extract 1,463 writing features. A non-linear regressor deep neural network is trained to predict holistic scores on a scale of 10–60. This study shows that deep learning holds the promise to improve significantly the accuracy of AES systems, but that the current dataset and most essay datasets fall short of providing them with enough expertise (hand-graded essays) to exploit that potential. After the tuning of different sets of hyperparameters, the results show that the levels of agreement, as measured by the quadratic weighted kappa metric, obtained on the training, validation, and testing sets are 0.84, 0.63, and 0.58, respectively, while an ensemble (bagging) produced a kappa value of 0.80 on the testing set. Finally, this paper upholds that more than 1,000 hand-graded essays per writing construct would be necessary to adequately train the predictive student models on automated essay scoring, provided that all score categories are equally or fairly represented in the sample dataset.

Open Research and Observational Study for 21 st Century Learning

Contemporary research practice unreasonably obscures formative research outcomes from public notice. Indeed, this exclusion – often unintentional – holds true even when the research is publicly funded. Accordingly, the Public must search scholarly channels, such as academic journals, for research information that is not composed for general comprehension. Essentially, a breach in information transmission separates researchers and society at large. In education, a similar communication gap exists between students and instructors, given that instructors rely on traditional assessment activities to measure student performance and rarely realize the corresponding study efforts. Consequently, certain important formative evidences go largely unnoticed. Today, researchers are exploring smart learning processes that exploit opportunities triggered by environmental affordance, personal need, and/or professional expectation, and mitigate various assessment difficulties.