Philip Butcher

Contexts for evaluating educational software

The evaluation of educational software is of concern to two particular academic communities: HCI and educational technology. There is a danger that usability features are considered at the expense of educational issues (and the converse of this is of course equally true). This paper considers how the notion and practice of evaluation in the educational community differs from that in HCI and also identifies areas of overlap. It then describes how particular influences and contexts have led one group of evaluators from the educational community to develop a context, interactions, attitudes and outcomes (CIAO!) model of evaluation for computer assisted learning (CAL) evaluation in distance education. The application of this model is illustrated by case studies from a recent evaluation project and related research. The paper concludes with a discussion of the issues raised for both communities by this model for evaluation. q 1999 Published by Elsevier Science B.V. All rights reserved.

Learning with computers: experiences of evaluation

Abstract At the Open University (OU), the perception of computer assisted learning (CAL) is shifting from its current position as an optional extra in a battery of course components to that of a key teaching medium. This paper presents the authors’ experiences of the evaluation of a range of current OU produced computer assisted learning materials, mainly but not exclusively in science and technology reported in terms of evaluation framework. These materials include interactive tutorial packages, simulations used largely in a support role to the main teaching and multimedia packages which are expected to carry more of the initial presentation of key ideas in the courses. The overall purpose of the evaluations was to gain insight into how computers can have an impact on learning, particularly of science and technology subjects, and to explore a portfolio of evaluation techniques to be used in the development and assessment of computer based learning materials. In this paper the techniques are discussed and illustrated with reference to a number of case studies of a range of programs used by students. Some conclusions are drawn about how the use of programs leads to the support of students’ science and technology learning.

Evaluating CAL at the open university: 15 years on

The paper reviews both the types of CAL in use at the Open University over the previous fifteen years and currently in the context of the literature on CAL in HE more generally. A framework for evaluating CAL is then proposed that focuses on Context, Interactions and Outcomes (CIAO) and an example of the evaluation approach in action is given.