Ole Andreas Alsos

Usability testing of mobile ICT for clinical settings: Methodological and practical challenges

BACKGROUND While much is known about how to do usability testing of stationary Electronic Patient Record (EPR) systems, less is known about how to do usability testing of mobile ICT systems intended for use in clinical settings. AIM Our aim is to provide a set of empirically based recommendations for usability testing of mobile ICT for clinical work. METHOD We have conducted usability tests of two mobile EPR systems. Both tests have been done in full-scale models of hospital settings, and with multiple users simultaneously. We report here on the methodological aspects of these tests. RESULTS We found that the usability of the mobile EPR systems to a large extent were determined by factors that went beyond that of the graphical user interface. These factors include ergonomic aspects such as the ability to have both hands free, and social aspects such as to what extent the systems disturbs the face-to-face interaction between the health worker and the patient. CONCLUSIONS To be able to measure usability issues that go beyond what can be found by a traditional stationary user interface evaluation, it is necessary to conduct usability tests of mobile EPR systems in physical environments that simulate the conditions of the work situation at a high level of realism. It is further in most cases necessary to test with a number of test subjects simultaneously.

Fidelity Considerations for Simulation-Based Usability Assessments of Mobile ICT for Hospitals

Controlled laboratory-based usability assessments of mobile information and communications technologies (ICT) for hospitals have been conducted. As part of these assessments, clinicians have acted out mobile work scenarios and used the systems to solve related tasks. The evaluations show that relevant usability issues go beyond those of graphical user interfaces. Many of these usability issues only show up when the real-world context of use is replicated in the laboratory to a high degree of fidelity. The complexity of the context of use for mobile ICT in hospitals has motivated us to explore training simulation fidelity theories. Based on a review of the training simulation literature, a set of fidelity dimensions through which training simulations are often adjusted to meet specific goals are identified. It is argued that the same mechanisms can be used in usability assessments of mobile ICT for hospitals. Our argument is substantiated by using the identified set of fidelity dimensions in a retrospective analysis of two usability assessments. The analysis explains how the configuration of fidelity dimensions, each reflecting various degrees of realism vis-à-vis the actual performance context, contributed to the identification of relevant usability issues.

Toward a best practice for laboratory-based usability evaluations of mobile ICT for hospitals

The dynamic nature of clinical work makes it challenging to assess the usability of mobile information and communication technology (ICT) for hospitals. To achieve some of the realism of field evaluations, combined with the control of laboratory-based evaluations, we have conducted usability tests of prototypes in a laboratory custom designed as a full-scale ward section. Nurses and physicians acting out simulated clinical scenarios have used the prototypes. This paper reports on the general methodological lessons learned from three such formative usability evaluations. We have learned that the physical test environment, the test scenarios, and the prototypes form three variables that need to reflect a sufficient amount of realism and concreteness in order to help generate valid test results. At the same time, these variables are tools that can help control and focus the evaluation on specific issues that one wants to gather data on. We have also learned that encouraging reflection among participants and using detailed multi-perspective recordings of usage can help form a more precise understanding of how mobile ICT can accommodate clinical work. The current paper aims to inform work toward a best practice for laboratory-based evaluations of mobile ICT for hospitals.

Designing for the secondary user experience

Computer systems are first and foremost designed for primary users. The needs of the other types of users, such as secondary users, are often overlooked. These users are not interacting with the system directly but are yet affected by it. This study is based on empirical findings from two usability evaluations in a realistic hospital setting with physicians and patient actors. We have found that also secondary users, such as patients, have a kind of user experience during the primary users interaction with the system. We conclude from this that designers and developers should also address the need of secondary users and include them in the design and evaluation process. This means designing devices or GUIs that (1) support non-verbal communication, (2) provide feedback to the secondary users, (3) use their language and representation, and (4) is tailored for the secondary user. Sometimes a focus on the secondary user implies that the designer must deal with conflicting needs between the primary and the secondary users.

Evaluating Mobile Usability: The Role of Fidelity in Full-Scale Laboratory Simulations with Mobile ICT for Hospitals

We have applied full-scale simulations to evaluate the usability of mobile ICT for hospitals in a realistic but controllable research setting. Designing cost-effective and targeted simulations for such a purpose raises the issue of simulation fidelity. Evaluators need to identify which aspects of the research setting that should appear realistic to simulation participants, and which aspect that can be removed or represented more abstractly. Drawing on research on training simulations, this paper discusses three interrelated fidelity components--equipment/prototype fidelity, environmental fidelity, and psychological fidelity. These components need to be adjusted according to which design aspects evaluators want to gather feedback on. We present examples of how we have configured the components in various simulation-based usability assessments of mobile ICT for hospitals. The paper concludes by providing a set of guiding principles concerning the role of fidelity in simulation-based usability evaluations.

Gamification and Family Housework Applications

This conceptual work represents an initial exploration into a little researched area, namely app design for families. We explore how gamification is incorporated in applications that target family housework, also known as chores. During the last five years an increasing number of apps aim to transform routine based housework into entertaining activities. Many parents think it is important that children, at an early age, learn about family values and responsibilities that comes with the role as a family member. However, a gamified approach towards housework can influence family interaction in both positive and negative ways. We analyze a selection of so-called chore apps by building on an existing classification framework for educational apps and applying concepts of game design elements. Our findings show (1) that existing apps tend to be mostly instructive and partly manipulable, (2) that they tend to focus on external rather than intrinsic motivation, (3) that they target family members individually, rather than the family as a whole. We discuss the results from a motivation perspective by drawing attention to three concepts that relate to intrinsic motivation: Competence, autonomy and relatedness.

Session-Aware Clinical Information Systems

Modern health information systems must be able to represent and reason about work-processes, roles, tasks and interaction in order to provide relevant and timely support. However, commonly used models of work and interaction does not address the details of everyday clinical collaborative communication. If we are to design clinical information systems that support processes, we have to deal with the reality as perceived by the clinician: A puzzle of information and communication about patients, decisions and events. We have identified that it is important to support and represent the dialogue between one or more users and systems.