Catherine Mulwa

A Web-Based Framework for User-Centred Evaluation of End-User Experience in Adaptive and Personalized E-Learning Systems

The evaluation of interactive adaptive and personalised systems has long been acknowledged as a difficult, complicated and very demanding Endeavour due to the complex nature of these systems. This paper describes a web-based framework for the evaluation of end-user experience in adaptive and personalised e-Learning systems. The benefits of the framework include: i) the provision of an interactive reference and recommendation tool to encourage the evaluation of systems that fulfil certain methodological requirements, ii) the collaborative nature of the framework facilitates the sharing of information among researchers from the information technology, adaptive hypermedia, information retrieval and e-Learning communities, iii) the identification of pitfalls in the evaluation planning process as well as in data analysis, and iv) the translation of presented information into users language of choice. This paper also presents a review of User-Centred Evaluation approaches, methodologies and techniques adopted by current systems and frameworks. The results of this review are analysed. From these results, an architectural design for the framework was specified.

A recommender framework for the evaluation of end user experience in adaptive technology enhanced learning

Adaptive Technology Enhanced Learning (TEL) has attracted significant interest with the promise of supporting individual learning tailored to the unique circumstances, preferences and prior knowledge of a learner. However, the evaluation of the overall performance of such systems is a major challenge, as the adaptive TEL system reacts differently for each individual user and context of use. Evaluation of such systems is significant but very complex area of research in itself since depending on the aspect of personalisation that needs to be evaluated. Several evaluation techniques need to be combined and executed differently. This paper proposed a novel recommender framework built upon an evaluation educational data set using a hybrid recommend approach to identify appropriate procedures. Recommendations are to software developers and users of adaptive TEL systems. A review and analyses of evaluation studies on adaptive TEL systems was conducted. Based on the analysed results, an educational evaluation data set was created.

A framework for the evaluation of adaptive IR systems through implicit recommendation

Personalised Information Retrieval (PIR) has gained considerable attention in recent literature. In PIR different stages of the retrieval process are adapted to the user, such as adapting the users query or the results. Personalised recommender frameworks are endowed with intelligent mechanisms to search for products, goods and services that users are interested in. The objective of such tools is to evaluate and filter the huge amount of information available within a specific scope to assist users in their information access processes. This paper presents a web-based adaptive framework for evaluating personalised information retrieval systems. The framework uses implicit recommendation to guide users in deciding which evaluation techniques, metrics and criteria to use. A task-based experiment was conducted to test the functionality and performance of the framework. A Review of evaluation techniques for personalised IR systems was conducted and the results of the analysed survey are presented.