Xavier Ochoa

Context-Aware Recommender Systems for Learning: A Survey and Future Challenges

Recommender systems have been researched extensively by the Technology Enhanced Learning (TEL) community during the last decade. By identifying suitable resources from a potentially overwhelming variety of choices, such systems offer a promising approach to facilitate both learning and teaching tasks. As learning is taking place in extremely diverse and rich environments, the incorporation of contextual information about the user in the recommendation process has attracted major interest. Such contextualization is researched as a paradigm for building intelligent systems that can better predict and anticipate the needs of users, and act more efficiently in response to their behavior. In this paper, we try to assess the degree to which current work in TEL recommender systems has achieved this, as well as outline areas in which further work is needed. First, we present a context framework that identifies relevant context dimensions for TEL applications. Then, we present an analysis of existing TEL recommender systems along these dimensions. Finally, based on our survey results, we outline topics on which further research is needed.

Quantitative Analysis of Learning Object Repositories

This paper conducts the first detailed quantitative study of the process of publication of learning objects in repositories. This process has been often discussed theoretically, but never empirically evaluated. Several question related to basic characteristics of the publication process are raised at the beginning of the paper and answered through quantitative analysis. To provide a wide view of the publication process, this paper analyzes four types of repositories: Learning Object Repositories, Learning Object Referatories, Open Courseware Initiatives, and Learning Management Systems. For comparison, Institutional Repositories are also analyzed. Three repository characteristics are measured: size, growth, and contributor base. The main findings are that the amount of learning objects is distributed among repositories according to a power law, the repositories mostly grow linearly, and the amount of learning objects published by each contributor follows heavy-tailed distributions. The paper finally discusses the implications that this findings could have in the design and operation of Learning Object Repositories.

Relevance Ranking Metrics for Learning Objects

This paper develops the concept of relevance in the context of learning object search. It proposes a set of metrics to estimate the topical, personal and situational relevance dimensions. These metrics are derived mainly from usage and contextual information. An exploratory evaluation of the metrics shows that even the simplest ones provide statistically significant improvement in the ranking order over the most common algorithmic relevance metric. Moreover, the combination of the metrics through the RankNet learning sorts the result list 50% better than the base-line ranking. The paper also presents open questions in the field of learning object relevance ranking that deserve further attention.

The Ariadne Infrastructure for Managing and Storing Metadata

Reusing digital resources for learning has been a goal for several decades, driven by potential time savings and quality enhancements. Although the rapid development of Web-based learning has increased opportunities for reuse significantly, managing learning objects and making them accessible still entails many challenges. This article presents and analyzes the standards-based Ariadne infrastructure for managing learning objects in an open and scalable architecture. The architecture supports the integration of learning objects in multiple, distributed repository networks. The authors capture lessons learned in four architectural patterns.

Use of contextualized attention metadata for ranking and recommending learning objects

The tools used to search and find Learning Objects in different systems do not provide a meaningful and scalable way to rank or recommend learning material. This work propose and detail the use of Contextual Attention Metadata, gathered from the different tools used in the lifecycle of the Learning Object, to create ranking and recommending metrics to improve the user experience. Four types of metrics are detailed: Link Analysis Ranking, Similarity Recommendation, Personalized Ranking and Contextual Recommendation. While designed for Learning Objects, it is shown that these metrics could also be applied to rank and recommend other types of reusable components like software libraries.

Automatic evaluation of metadata quality in digital repositories

Owing to the recent developments in automatic metadata generation and interoperability between digital repositories, the production of metadata is now vastly surpassing manual quality control capabilities. Abandoning quality control altogether is problematic, because low-quality metadata compromise the effectiveness of services that repositories provide to their users. To address this problem, we present a set of scalable quality metrics for metadata based on the Bruce & Hillman framework for metadata quality control. We perform three experiments to evaluate our metrics: (1) the degree of correlation between the metrics and manual quality reviews, (2) the discriminatory power between metadata sets and (3) the usefulness of the metrics as low-quality filters. Through statistical analysis, we found that several metrics, especially Text Information Content, correlate well with human evaluation and that the average of all the metrics are roughly as effective as people to flag low-quality instances. The implications of this finding are discussed. Finally, we propose possible applications of the metrics to improve tools for the administration of digital repositories.

On the use of learning object metadata: the GLOBE experience

Since IEEE LTSC LOM was published in 2002, it is one of the widest adopted standard for the description of educational resources. The GLOBE (Global Learning Objects Brokered Exchange) alliance enables share and reuse between several Learning Object Repositories worldwide. Being the largest and more diverse collection of Learning Object Metadata, it is an ideal place to perform an analysis of the actual use of the LOM standard in the real world. This paper presents an in-depth analysis of the use and quality of 630.317 metadata instances.

Expertise estimation based on simple multimodal features

Multimodal Learning Analytics is a field that studies how to process learning data from dissimilar sources in order to automatically find useful information to give feedback to the learning process. This work processes video, audio and pen strokes information included in the Math Data Corpus, a set of multimodal resources provided to the participants of the Second International Workshop on Multimodal Learning Analytics. The result of this processing is a set of simple features that could discriminate between experts and non-experts in groups of students solving mathematical problems. The main finding is that several of those simple features, namely the percentage of time that the students use the calculator, the speed at which the student writes or draws and the percentage of time that the student mentions numbers or mathematical terms, are good discriminators be- tween experts and non-experts students. Precision levels of 63% are obtained for individual problems and up to 80% when full sessions (aggregation of 16 problems) are analyzed. While the results are specific for the recorded settings, the methodology used to obtain and analyze the features could be used to create discriminations models for other contexts.

Presentation Skills Estimation Based on Video and Kinect Data Analysis

This paper identifies, by means of video and Kinect data, a set of predictors that estimate the presentation skills of 448 individual students. Two evaluation criteria were predicted: eye contact and posture and body language. Machine-learning evaluations resulted in models that predicted the performance level (good or poor) of the presenters with 68% and 63% of correctly classified instances, for eye contact and postures and body language criteria, respectively. Furthermore, the results suggest that certain features, such as arms movement and smoothness, provide high significance on predicting the level of development for presentation skills. The paper finishes with conclusions and related ideas for future work.

Semi-automatic assembly of learning resources

Technology Enhanced Learning is a research field that has matured considerably over the last decade. Many technical solutions to support design, authoring and use of learning activities and resources have been developed. The first datasets that reflect the tracking of actual use of these tools in real-life settings are beginning to become available. In this article, we present an exploratory study that relies on these datasets to support semi-automatic assembly of learning activities and resources for specific contexts. Starting from learning designs and other online sources that describe well designed learning experiences as they were used in practice, we derive sequencing patterns that capture re-occurring patterns of activities. A semi-automatic assembly framework uses these patterns to support teachers in the design and authoring of course activities. We present a case study that integrates recommendation support for sequencing activities as well as associated resources in the LAMS learning activity environment. Results indicate that the perceived usefulness is high: both teachers with expertise in the use of learning design tools as well as teachers with no background knowledge in the area indicate that the recommendations helped them in the authoring process. In addition, they feel more confident using learning design tools when support is provided that is driven by best practice knowledge.