Yngve Dahl

Usability testing of mobile ICT for clinical settings: Methodological and practical challenges

BACKGROUND While much is known about how to do usability testing of stationary Electronic Patient Record (EPR) systems, less is known about how to do usability testing of mobile ICT systems intended for use in clinical settings. AIM Our aim is to provide a set of empirically based recommendations for usability testing of mobile ICT for clinical work. METHOD We have conducted usability tests of two mobile EPR systems. Both tests have been done in full-scale models of hospital settings, and with multiple users simultaneously. We report here on the methodological aspects of these tests. RESULTS We found that the usability of the mobile EPR systems to a large extent were determined by factors that went beyond that of the graphical user interface. These factors include ergonomic aspects such as the ability to have both hands free, and social aspects such as to what extent the systems disturbs the face-to-face interaction between the health worker and the patient. CONCLUSIONS To be able to measure usability issues that go beyond what can be found by a traditional stationary user interface evaluation, it is necessary to conduct usability tests of mobile EPR systems in physical environments that simulate the conditions of the work situation at a high level of realism. It is further in most cases necessary to test with a number of test subjects simultaneously.

Fidelity Considerations for Simulation-Based Usability Assessments of Mobile ICT for Hospitals

Controlled laboratory-based usability assessments of mobile information and communications technologies (ICT) for hospitals have been conducted. As part of these assessments, clinicians have acted out mobile work scenarios and used the systems to solve related tasks. The evaluations show that relevant usability issues go beyond those of graphical user interfaces. Many of these usability issues only show up when the real-world context of use is replicated in the laboratory to a high degree of fidelity. The complexity of the context of use for mobile ICT in hospitals has motivated us to explore training simulation fidelity theories. Based on a review of the training simulation literature, a set of fidelity dimensions through which training simulations are often adjusted to meet specific goals are identified. It is argued that the same mechanisms can be used in usability assessments of mobile ICT for hospitals. Our argument is substantiated by using the identified set of fidelity dimensions in a retrospective analysis of two usability assessments. The analysis explains how the configuration of fidelity dimensions, each reflecting various degrees of realism vis-à-vis the actual performance context, contributed to the identification of relevant usability issues.

A comparison of location and token-based interaction techniques for point-of-care access to medical information

This paper compares the usability of some location and token-based interaction techniques for systems that provide point-of-care access to medical information. The investigation is based around a scenario from clinical work—administration of medicine to patients. Four interaction techniques that match the scenario are identified. We demonstrate how these techniques can be concretized through functional prototypes. The prototypes were tested with health workers in a full-scale model of a section of a hospital ward. The usability issues emerging from the tests were related to required user attention, predictability of system behavior, and integration with the work situation. We found that the usability of the interaction techniques to a large degree depended on specific physical and social conditions of the use situation. This result is an incentive to consider a broad set of sensor-based interaction techniques and devices for such systems, and to select the best few of these for implementation.

Toward a best practice for laboratory-based usability evaluations of mobile ICT for hospitals

The dynamic nature of clinical work makes it challenging to assess the usability of mobile information and communication technology (ICT) for hospitals. To achieve some of the realism of field evaluations, combined with the control of laboratory-based evaluations, we have conducted usability tests of prototypes in a laboratory custom designed as a full-scale ward section. Nurses and physicians acting out simulated clinical scenarios have used the prototypes. This paper reports on the general methodological lessons learned from three such formative usability evaluations. We have learned that the physical test environment, the test scenarios, and the prototypes form three variables that need to reflect a sufficient amount of realism and concreteness in order to help generate valid test results. At the same time, these variables are tools that can help control and focus the evaluation on specific issues that one wants to gather data on. We have also learned that encouraging reflection among participants and using detailed multi-perspective recordings of usage can help form a more precise understanding of how mobile ICT can accommodate clinical work. The current paper aims to inform work toward a best practice for laboratory-based evaluations of mobile ICT for hospitals.

Designing Pervasive Computing for Hospitals: Learning from the Media Affordances of Paper-Based Medication Charts

The current article investigates the affordances associated with paper out of the motivation that this can help inform design of pervasive and ubiquitous computing solutions for clinical use. In particular, we focus on paper-based medication charts, and discuss how these artifacts differ from various digital and context-aware solutions both in terms of media properties and functionalities. Based on a comparison of media, we argue that the affordances offered by the paper-based medication charts are not fully transferable to one digital medium, but that a combination of complementary digital media is required

Stakeholder Attitudes Toward and Values Embedded in a Sensor-Enhanced Personal Emergency Response System

This paper provides an empirical understanding of concerns that the application of a sensorenhanced medical alert system, or personal emergency response (PER) system, raises from the perspective of care receivers (users) and care providers. Data were gathered in the context of a field trial of a PER system supporting both user-initiated alerts and automatic fall detection alerts. The system was tested at two residential care facilities for 3 weeks. Drawing on data primarily from post-trial group and pair interviews, we describe and compare care receivers’ and providers’ views on the following emerging concerns: (i) form factor and ergonomics, (ii) system feedback and user control and (iii) sensor precision and trust. Based on feedback from stakeholder groups, we discuss potential value biases, or discriminating factors, embedded in the evaluated PER system. We also discuss the implications of our findings for a value-driven design agenda for future PER systems.

Designing a reliable pain drawing tool: avoiding interaction flaws by better tailoring to patients' impairments

AbstractPatients with advanced cancer are influenced by the disease itself and by treatment side effects, both of which may have great impact on their lives. One of the most distressing symptoms is pain. However, pain in cancer patients can in most cases be relieved if the patient is able to communicate the nature and severity of the problem to the healthcare professionals through an effective assessment process. The main goal of this paper is to help form an understanding of central patient characteristics that should be taken into account when designing pain assessment tools for patients with advanced cancer. Traditionally, pain has been assessed by paper-based questionnaires and pain drawings. An iterative study was conducted based on repeated cycles of usability testing of a computerized pain body map for communicating pain by advanced cancer patients. Our aim was to provide a patient interface that most patients were able to interact with, collecting valuable, granular pain information with a minimum of strain on the patient. Through this process, we identified and solved design issues related to the sickest and frailest cancer patients. We further created a web-based solution for collecting individual pain drawings for evaluation by clinicians. The concept was appreciated by the patients, and the information provided was considered valuable by physicians. The main contribution of this paper is a list of suggestions to guide the design of an interactive tool for patients with advanced cancer.

Co-designing interactive tabletop solutions for active patient involvement in audiological consultations

Active patient participation in audiological treatment is central in improving the benefits and satisfaction that patients experience in the use of hearing aids. However, involving patients in their own treatment is challenging. Clinics often lack appropriate tools for information sharing and collaboration between patients and audiologists. In addition, patients may find it difficult to relate the treatment provided at the clinic to the hearing and hearing aid problems they experience in their daily listening environments. To help remedy these challenges we have investigated how hearing aid users and audiologists look on the idea of using shared tabletop interfaces, in combination with simulated sound environments, to support active patient participation during hearing aid adjustments. We have conducted three workshops in which hearing aid users and audiologists have co-designed low-fidelity mock-ups of tabletop interfaces. This paper reports on central design concepts and considerations that emerged through this work, and discusses the main design lessons that can be learned from this.

Designing Smart Home Technology for Fall Prevention in Older People

Falls in older people constitute one of the major challenges in healthcare. It is important to design technologies that can help prevent falls and improve falls management. Smart home technology could be of importance in this context, but the technology has to be user-centred or adapted to be useful in this particular context. This study assessed usability of paper and interactive prototypes of a smart home touch screen panel. The study implemented five scenarios related to fall risk, fall assessment and exercise guidance, designing a smart home interface for independent living in general and fall management in particular. A usability evaluation showed that older people had positive experiences when using the touch screen interface. The study demonstrated the need for user-centred interfaces for older people in the context of falls prevention.

Evaluating Mobile Usability: The Role of Fidelity in Full-Scale Laboratory Simulations with Mobile ICT for Hospitals

We have applied full-scale simulations to evaluate the usability of mobile ICT for hospitals in a realistic but controllable research setting. Designing cost-effective and targeted simulations for such a purpose raises the issue of simulation fidelity. Evaluators need to identify which aspects of the research setting that should appear realistic to simulation participants, and which aspect that can be removed or represented more abstractly. Drawing on research on training simulations, this paper discusses three interrelated fidelity components--equipment/prototype fidelity, environmental fidelity, and psychological fidelity. These components need to be adjusted according to which design aspects evaluators want to gather feedback on. We present examples of how we have configured the components in various simulation-based usability assessments of mobile ICT for hospitals. The paper concludes by providing a set of guiding principles concerning the role of fidelity in simulation-based usability evaluations.