Erica Morris

Contexts for evaluating educational software

The evaluation of educational software is of concern to two particular academic communities: HCI and educational technology. There is a danger that usability features are considered at the expense of educational issues (and the converse of this is of course equally true). This paper considers how the notion and practice of evaluation in the educational community differs from that in HCI and also identifies areas of overlap. It then describes how particular influences and contexts have led one group of evaluators from the educational community to develop a context, interactions, attitudes and outcomes (CIAO!) model of evaluation for computer assisted learning (CAL) evaluation in distance education. The application of this model is illustrated by case studies from a recent evaluation project and related research. The paper concludes with a discussion of the issues raised for both communities by this model for evaluation. q 1999 Published by Elsevier Science B.V. All rights reserved.

Guest Editorial:Context, collaboration, computers and learning

E. Morris & R. Joiner In October 2000, the Computers and Learning Research Group (CALRG) at the Open University held a Symposium on Computer-Supported Collaborative Learning (CSCL). The CALRG aims to support interdisciplinary research by holding symposia and an annual conference to encourage discussion and debate, and the exchange of information on current research work. The rationale for the symposium was that much work in the area of CSCL had been done by members of the Group and there was a need to look at current projects and activities involving members at the OU and researchers from other institutions. Previous research work involved both adults and children in schools and investigated computer-supported collaborative learning in the area of science education (Scanlon, 1995). In recent years, CALRG had a programme of work with an interrelated set of projects that had the common aim of describing how ICT can provide learners with productive experiences of collaborative learning using a variety of computer systems. For example, a Gameshow project looked at collaboration in the context of adults working on a shared simulation of a statistics problem which was implemented in a distributed classroom environment (Joiner et al., 2001). The papers in this Special Issue are the outcome of the symposium and provide a stimulating picture of current work in the area of CSCL. Initially, research in this area was concerned with the preconditions and consequences of computer-based collaborative activity on individuals’ cognitive behaviour. Collaboration was viewed as the ‘mutual engagement of participants in a co-ordinated effort to solve the problem together’ (Roschelle & Teasley, 1995). Research now tends to focus on the context of collaborative activity, uses broader definitions of collaboration and investigates it in a wider range of settings. The following papers address these issues. The first two papers report studies that concern how the social setting alters the nature of collaborative activity. Hammond and Bennett explore how academic disciplines differ in their use of ICT to support small group activities in higher education. They present a framework for examining discipline differences and present evidence from surveys and case studies conducted by the ASTER project. The authors use a framework as a means of describing and identifying those differences and use this to examine the use of ICT to support small group learning in three disciplines: physics, psychology and humanities. The most striking difference they found was that between the physical sciences and the other disciplines. Humanities and psychology used ICT to support discussion, whereas there were no instances of this use of ICT in physics. All ICT use in physics focussed on task-based activities. This difference, the authors argue, reflects a difference in underlying concepts and representations between the disciplines and/or a difference in discipline beliefs and assumptions. The authors argue that it is vital that such differences are considered when advising on the take-up of innovative teaching and learning methods.

Use of Concept Maps to Represent Students' Knowledge of Research Methods in Psychology: A Preliminary Study

This paper is a report of research on the potential validity of concept maps as a representation of students’ knowledge of research methods in psychology. Concept maps may provide insight into the content and structure of students’ knowledge that traditional assessment methods do not. In this paper we examine the rationale underlying the use of concept maps and issues associated with their use. We also report a qualitative study in which we employed concept maps to assess students’ knowledge. Finally, there is a description of a potential application of concept mapping to the evaluation of students’ knowledge, currently being investigated by the authors.