Stephan Weibelzahl

Developing Adaptive Internet Based Courses with the Authoring System NetCoach

Developing adaptive internet based learning courses usually requires a lot of programming efforts to provide session management, keeping track of the learners current state, and adapting the interface layout to specific requirements. NetCoach is designed to enable authors to develop adaptive learning courses without programming knowledge. In this paper, we describe the adaptive, the adaptable, the interactive, and the communicative features of NetCoach. Both authors and tutors are supported in many ways to develop and manage courses via an online interface. Experiences with NetCoach courses in different domains and settings have shown that learners profit from the adaptive features.

Evaluation of Adaptive Systems

Unambiguously, adaptive systems have to be evaluated empirically to guarantee that the adaptivity really works. Nevertheless, only few of the existing adaptive systems have been evaluated. One of the most important reasons for this lack is, that measures for adaptivity success have not been investigated systematically up to now. The aim of this PhD thesis is to explore a methodology for the empirical evaluation of adaptive systems, including validated criteria, experimental designs and procedures. It will be demonstrated that cognitive and behavioral factors provide important evidence for adaptivity success.

Layered evaluation of interactive adaptive systems: framework and formative methods

The evaluation of interactive adaptive systems has long been acknowledged to be a complicated and demanding endeavour. Some promising approaches in the recent past have attempted tackling the problem of evaluating adaptivity by “decomposing” and evaluating it in a “piece-wise” manner. Separating the evaluation of different aspects can help to identify problems in the adaptation process. This paper presents a framework that can be used to guide the “layered” evaluation of adaptive systems, and a set of formative methods that have been tailored or specially developed for the evaluation of adaptivity. The proposed framework unifies previous approaches in the literature and has already been used, in various guises, in recent research work. The presented methods are related to the layers in the framework and the stages in the development lifecycle of interactive systems. The paper also discusses practical issues surrounding the employment of the above, and provides a brief overview of complementary and alternative approaches in the literature.

Usability engineering for the adaptive web

This chapter discusses a usability engineering approach for the design and the evaluation of adaptive web-based systems, focusing on practical issues. A list of methods will be presented, considering a user-centered approach. After having introduced the peculiarities that characterize the evaluation of adaptive web-based systems, the chapter describes the evaluation methodologies following the temporal phases of evaluation, according to a user-centered approach. Three phases are distinguished: requirement phase, preliminary evaluation phase, and final evaluation phase. Moreover, every technique is classified according to a set of parameters that highlight the practical exploitation of that technique. For every phase, the appropriate techniques are described by giving practical examples of their application in the adaptive web. A number of issues that arise when evaluating an adaptive system are described, and potential solutions and workarounds are sketched.

Log file analysis for disengagement detection in e-Learning environments

Most e-Learning systems store data about the learner’s actions in log files, which give us detailed information about learner behaviour. Data mining and machine learning techniques can give meaning to these data and provide valuable information for learning improvement. One area that is of particular importance in the design of e-Learning systems is learner motivation as it is a key factor in the quality of learning and in the prevention of attrition. One aspect of motivation is engagement, a necessary condition for effective learning. Using data mining techniques for log file analysis, our research investigates the possibility of predicting users’ level of engagement, with a focus on disengaged learners. As demonstrated previously across two different e-Learning systems, HTML-Tutor and iHelp, disengagement can be predicted by monitoring the learners’ actions (e.g. reading pages and taking test/quizzes). In this paper we present the findings of three studies that refine this prediction approach. Results from the first study show that two additional reading speed attributes can increase the accuracy of prediction. The second study suggests that distinguishing between two different patterns of disengagement (spending a long time on a page/test and browsing quickly through pages/tests) may improve prediction in some cases. The third study demonstrates the influence of exploratory behaviour on prediction, as most users at the first login familiarize themselves with the system before starting to learn.

A decomposition model for the layered evaluation of interactive adaptive systems

A promising approach towards evaluating adaptive systems is to decompose the adaptation process and evaluate the system in a “piece-wise” manner. This paper presents a decomposition model that integrates two previous proposals. The main “stages” identified are: (a) collection of input data, (b) interpretation of the collected data, (c) modeling of the current state of the “world”, (d) deciding upon adaptation, and (e) applying adaptation.

Cross-system validation of engagement prediction from log files

Engagement is an important aspect of effective learning. Time spent using an e-Learning system is not quality time if the learner is not engaged. Tracking the student disengagement would give the possibility to intervene for motivating the learner at appropriate time. In previous research we showed the possibility to predict engagement from log files using a web-based e-Learning system. In this paper we present the results obtained from another web-based system and compare them to the previous ones. The similarity of results across systems demonstrates that our approach is system-independent and that engagement can be elicited from basic information logged by most e-Learning systems: number of pages read, time spent reading pages, number of tests/ quizzes and time spent on test/ quizzes.

Eliciting Motivation Knowledge from Log Files Towards Motivation Diagnosis for Adaptive Systems

Motivation is well-known for its importance in learning and its influence on cognitive processes. Adaptive systems would greatly benefit from having a user model of the learners motivation, especially if integrated with information about knowledge. In this paper a log file analysis for eliciting motivation knowledge is presented, as a first step towards a user model for motivation. Several data mining techniques are used in order to find the best method and the best indicators for disengagement prediction. Results show a very good level of prediction: around 87% correctly predicted instances of all three levels of engagement and 93% correctly predicted instances of disengagement. Data sets with reduced attribute sets show similar results, indicating that engagement level can be predicted from information like reading pages and taking tests, which are common to most e-Learning systems.

User-centered EEG-based multimedia quality assessment

Multimedia users are becoming increasingly quality-aware as the technological advances make ubiquitous the creation and delivery of high-definition multimedia content. While much research work has been conducted on multimedia quality assessment, most of the existing solutions come with their own limitations, with particular solutions being more suitable to assess particular aspects related to users Quality of Experience (QoE). In this context, there is an increasing need for innovative solutions to assess users QoE with multimedia services. This paper proposes the QoE-EEG-Analyser that provides a solution to automatically assess and quantify the impact of various factors contributing to users QoE with multimedia services. The proposed approach makes use of participants frustration level measured with a consumer-grade EEG system, the Emotiv EPOC. The main advantage of QoE-EEG-Analyser is that it enables continuous assessment of various QoE factors over the entire testing duration, in a non-invasive way, without requiring the user to provide input about his perceived visual quality. Preliminary subjective results have shown that frustration can indicate users perceived QoE.

Integration of e-learning and knowledge management – barriers, solutions and future issues

The findings of the Workshop on Learner-oriented Knowledge Management and KM-oriented e-Learning (LOKMOL 2005) are summarized in this paper. The results are derived from the presented papers as well as from the moderated discussion during the workshop. First, the main barriers that have to be passed in order to integrate KM and e-Learning are discussed. Secondly, the approaches and technologies of the LOKMOL contributions are summarized and thirdly we provide issues that should be addressed in the future in order to successfully integrate KM and e-Learning.