Maria Francesca Costabile

On the Usability Evaluation of E-Learning Applications

Despite the advances of the electronic technologies in e-learning, a consolidated evaluation methodology for e-learning applications does not yet exist. The goal of e-learning is to offer the users the possibility to become skillful and acquire knowledge on a new domain. The evaluation of educational software must consider its pedagogic effectiveness as well as its usability. The design of its interface should take into account the way students learn and also provide good usability so that students interactions with the software are as natural and intuitive as possible. In this paper, we present the results obtained from a first phase of observation and analysis of the interactions of people with e-learning applications. The aim is to provide a methodology for evaluating such applications.

On the Advantages of a Systematic Inspection for Evaluating Hypermedia Usability

It is indubitable that usability inspection of complex hypermedia is still an art, in the sense that a great deal is left to the skills, experience, and ability of the inspectors. Training inspectors is difficult and often quite expensive. The Systematic Usability Evaluation (SUE) inspection technique has been proposed to help usability inspectors share and transfer their evaluation know-how, to simplify the hypermedia inspection process for newcomers, and to achieve more effective and efficient evaluation results. SUE inspection is based on the use of evaluation patterns, called abstract tasks, which precisely describe the activities to be performed by evaluators during inspection. This article highlights the advantages of this inspection technique by presenting its empirical validation through a controlled experiment. Two groups of novice inspectors were asked to evaluate a commercial hypermedia CD-ROM by applying the SUE inspection or traditional heuristic evaluation. The comparison was based on three major dimensions: effectiveness, efficiency, and satisfaction. Results indicate a clear advantage of the SUE inspection over the traditional inspection on all dimensions, demonstrating that abstract tasks are efficient tools to drive the evaluators performance.

Systematic evaluation of e-learning systems: an experimental validation

The evaluation of e-learning applications deserves special attention and evaluators need effective methodologies and appropriate guidelines to perform their task. We have proposed a methodology, called eLSE (e-Learning Systematic Evaluation), which combines a specific inspection technique with user-testing. This inspection aims at allowing inspectors that may not have a wide experience in evaluating e-learning systems to perform accurate evaluations. It is based on the use of evaluation patterns, called Abstract Tasks (ATs), which precisely describe the activities to be performed during inspection. For this reason, it is called AT inspection. In this paper, we present an empirical validation of the AT inspection technique: three groups of novice inspectors evaluated a commercial e-learning system applying the AT inspection, the heuristic inspection, or user-testing. Results have shown an advantage of the AT inspection over the other two usability evaluation methods, demonstrating that Abstract Tasks are effective and efficient tools to drive evaluators and improve their performance. Important methodological considerations on the reliability of usability evaluation techniques are discussed.

Visual metaphors for interacting with databases

The need for better human-computer interaction (HCI) has been widely recognized and discussed, even in the database area. It is generally accepted that the quality of the interaction mainly depends on the interface characteristics. How does one recognize good interfaces? The use of a suitable metaphor is crucial. Unfortunately, to put it as a metaphor, speaking about metaphors in HCI is like walking on a slippery floor.We would like to come up with a definition of metaphor that is specific to the particular needs of the database area, and, of course, of the database users. Thus, we will highlight some peculiarities of the database interaction. The following considerations should constitute a basis towards a formal approach for the construction of effective metaphors for interacting with databases.

Comparing low cost input devices for interacting with 3D Virtual Environments

Interaction with 3D Virtual Environments has always suffered from a lack of widely available and low cost input devices. Recently, thanks to the diffusion of gaming systems such as the Microsoft XBox 360 or the Nintendo Wii, new input devices are on the market at a relatively cheap price. This paper describes a study whose aim is to compare input devices in order to identify effective alternatives for the mouse and keyboard in such settings where their use is not advisable or feasible, e.g. museums and other public areas. This study has been carried out using a 3D Virtual Environment in which the participants were required to perform three canonical 3D interaction tasks. Two different groups participated to the test: the first group was involved in a pilot study to check the test environment. The second group performed the test.

A visual approach to HCI

The Pictorial Computing Laboratory (PCL) evolves its past experience in image processing and pattern recognition to the design of interactive systems. In the last ten years, a model for visual interactive computing has been developed based on the following abstraction: in interactive activities human beings communicate with computers through digital messages representing (part of) the state of the computation. Such a model, called Com2, is described, and an interaction with a system, developed following it, is shown as example, thus highlighting the use of visual languages for human-computer interaction.

Specification of visual languages as means for interaction

Interactive and end-user computing are the present challenges of computer science. They suggest the need for providing a specification methodology driving the development of effective interactive visual systems. In this line, we discuss a definition of visual languages which binds the interactive and communicative aspects with the computational ones, thus integrating a theory of visual languages with interaction-related aspects. By adopting this novel point of view, it becomes possible to study visual sentences and visual languages with respect to the basic principle of interaction that the users cannot cause unexpected events or become disoriented by the system behaviour. A classification of visual languages based on their capability of respecting this principle is then discussed and existing visual languages are framed in the classification.

A multiparadigmatic environment for interacting with databases

We present a prototype system to be used for visually accessing heterogeneous databases. The basic idea is to provide the user with several visual representations of data as well as multiple interaction mechanisms for both querying databases and visualizing the query results. Since some visual representations better fit certain user classes, the system adapts to the users needs by switching to the most appropriate visual representation and interaction mechanism, according to a suitable user model. The data and query representations are consistent in every paradigm. Such a notion of consistency stems from formal basis, i.e. a twofold data/representation model, namely the Graph Model, and a set of basic Graphical Primitives. This formal approach allows the user to switch from one interaction paradigm to another, always saving the query state.

Guiding usability evaluators during hypermedia inspection

This paper presents an empirical validation of the abstract tasks (ATs), which are operational guidelines driving the inspection activities during a usability evaluation. Two groups of inspectors evaluated a multimedia CD-ROM, one group using the ATs, the other group using a list of heuristics. Results demonstrated a better performance of the AT group but raised some issues concerning the acceptability of the technique.

Proposing Guidelines for Usability Inspection

Guidelines for designing interactive systems are largely available today in HCI literature. They provide the designer with a basis for making good decisions. Applying guidelines is the starting point towards good design. However, it is generally acknowledged that the only way to actually determine the quality of the obtained interactive systems is to perform usability evaluation. Our research is aimed at developing effective evaluation methods, as solicited by the increasing demand of cost-effective usability evaluation tools, which especially comes from industry. In this paper we describe a novel inspection technique we have defined. It is based on the use of operational guidelines, called Abstract Tasks, that systematically drive the inspection activities, thus allowing even less experienced evaluators to come up with valuable results.