Pekka Reijonen

Taking organizational implementation seriously: the case of IOS implementation

Despite of the rapid technical development, failures in information systems implementation are common and it seems obvious that the implementation of inter-organizational systems (IOS) include all the same possibilities for failures as intra-organizational systems - and unfortunately even some more. In this paper, we present some empirically proven means for avoiding problems during the implementation of IOSs. Our argumentation is based on the idea of organizational implementation of information systems, where the phases before and after the technical implementation are considered to be the most critical ones. The data from a case study are used to illustrate and support the ideas presented.

Back to User-Centered Usability Testing

Usability testing is a widely used evaluation method for product design during and after the development. Conventional usability testing applies short and discrete test tasks and task scenarios that are based on the tasks the product is designed to support. Thus, conventional test task design relies heavily on the representations of the specified context of use and the specified user requirements of the proposed design solution. However, a premature commitment to the specified context, requirements and proposed solutions may limit the scope of usability testing in a manner that hinders its capability to elicit and validate new user requirements, which is one of the objectives of the evaluation phase in the iterative user-centered design process. In this paper, we introduce a user-centered task design approach, which allows test participants to follow their natural work flow and freely express their needs during a test session. The main idea of this open-ended task approach is to break the tight link between the produced design solutions and the tasks used in the usability test and in this way increase the probability that novel user needs can emerge during a test session. Empirical results from a case study are used to depict the approach and its prerequisites, strengths, and limitations are discussed.

Visual simulation as an aid to understanding computer functions

Abstract A help-system with a visual simulation model is presented and evaluated. The simulation model allows the user to follow the transactions in a storage department on different hierarchical levels of detail; transaction flows between departments and units, detailed manipulation on the section level, and step-by-step progress of computer functions. The time dimension is taken into consideration, and the transactions can be followed forwards but may also be traced backwards. The backward trace is accomplished by two history files concerning data and actions taken. The model is designed to be used in two modes: simulation controlled by the system and simulation controlled by the user, controlled and interactive simulation, respectively. A prototype version of controlled simulation is presented as an example. Performed experiments show that the simulation model is useful as an aid in learning to understand computer functions.

Artifacts, Tools and Generalizing Usability Test Results

Usability testing has gained a rather stable status as a method for usability evaluation even though it has both low reliability and validity. The sources of result variance are well acknowledged among researchers and practitioners. However, the validity problem has not been explicated or exemplified although it is frequently discussed in the literature how the results of usability tests should be interpreted and to what extent results are generalizable. We employ Activity Theory and a case example to argue that the validity problem is mainly caused by the fact that what we are testing are artifacts and what people are using in their real life activities are tools and these two entities are qualitatively different. Basing on our analysis, the effects of the reliability and validity problems on the application of usability testing and its role as one of the tools in the design process are discussed.

Testing the Unknown – Value of Usability Testing for Complex Professional Systems Development

To make an impact on the design in usability testing, the test tasks are essential ingredients for the early system development process. Complex design problems are not solved by focusing on the details of a prototype and setting the scope on what is already known by the design team. Instead, the design value of usability testing is increased by deliberately relinquishing the assumptions made and implemented into a design. In the development of complex systems, usability testing with extended scope and open-ended structure, as presented in this paper with three empirical cases, delivers not only specific knowledge about the user interactions with the system, but reveals issues that, despite rigorous user research efforts, have been overlooked in the preceding phases of system development. Therefore, we suggest applying open-ended usability test tasks for testing systems in complex settings such as in the development of health care systems.

Are We Testing Utility? Analysis of Usability Problem Types

Usability problems and related redesign recommendations are the main outcome of usability tests although both are questioned in terms of impact in the design process. Problem classifications aim to provide better feedback for designers by improving usability problem identification, analysis and reporting. However, within the classifications, quite little is discussed about the types and the contents of usability problems as well as the types of required design efforts. We address this problem by scrutinizing the findings of three empirical usability tests conducted in software development projects. As a result, 173 problems were classified into 11 categories. Specific focus was placed on the distinction between the utility and usability types of problems, in order to define the correct development phase and method to fix the problem. The number of utility problems varied from 51 % to 74 %, which shows that early usability testing with a think-aloud protocol and an open task structure measure both utility and usability equally well.

Trends in media-based learning: personal computer hypermedia

The current emergence of optical memory media, especially in the form of CD-ROM technology, is changing the technology platform of media-based education. It is possible to design complex hypermedia learning environments, utilizing only a standard personal computer, as is shown in the work of the present authors. The psychological benefits of media-based learning are discussed.