Mikael B. Skov

Is It Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field

Evaluating the usability of mobile systems raises new concerns and questions, challenging methods for both lab and field evaluations. A recent literature study showed that most mobile HCI research projects apply lab-based evaluations. Nevertheless, several researchers argue in favour of field evaluations as mobile systems are highly context-dependent. However, field-based usability studies are difficult to conduct, time consuming and the added value is unknown. Contributing to this discussion, this paper compares the results produced by a laboratory- and a field-based evaluation of the same context-aware mobile system on their ability to identify usability problems. Six test subjects used the mobile system in a laboratory while another six used the system in the field. The results show that the added value of conducting usability evaluations in the field is very little and that recreating central aspects of the use context in a laboratory setting enables the identification of the same usability problem list.

You can touch, but you can't look: interacting with in-vehicle systems

Car drivers are nowadays offered a wide array of in-vehicle systems i.e. route guidance systems, climate controls, music players. Such in-vehicle systems often require the drivers visual attention, but visual workload has shown significant less eyes-on-the-road time and affects driving performance. In this paper, we illustrate and compare three different interaction techniques for in-vehicle systems. We refer to them as tactile, touch, and gesture interaction. The focus of the techniques is the effects on drivers while driving cars. We evaluated the interaction techniques with 16 subjects in two settings. Our results showed that gesture interaction has a significant effect on the number of driver eye glances especially eye fixations of more seconds. However, gesture interaction still required rapid eye glances for hand/eye coordination. On the other hand, touch interaction leads to fast and efficient task completion while tactile interaction seemed inferior to the two other interaction techniques.

A review of research methods in children's technology design

Research methods have been objects of discussions for decades and defining research methods is still a quite considerable challenge. However, it is important to understand research methods in different disciplines as it informs us on future directions and influences on the discipline.We conduct a survey of research methods in paper publications. 105 papers on childrens technology design are classified on a two-dimensional matrix on research method and purpose. Our results show a strong focus on engineering of products as applied research and on evaluation of developed products in the field or in the lab. Also, we find that much research is conducted in natural setting environments with strong focus on field studies.

Using mobile phones to support sustainability: a field study of residential electricity consumption

Recent focus on sustainability has made consumers more aware of our joint responsibility for conserving energy resources such as electricity. However, reducing electricity use can be difficult with only a meter and a monthly or annual electricity bill. With the emergence of new power meters units, information on electricity consumption is now available digitally and wirelessly. This enables the design and deployment of a new class of persuasive systems giving consumers insight into their use of energy resources and means for reducing it. In this paper, we explore the design and use of one such system, Power Advisor, promoting electricity conservation through tailored information on a mobile phone or tablet. The use of the system in 10 households was studied over 7 weeks. Findings provide insight into peoples awareness of electricity consumption in their home and how this may be influenced through design.

Comparison of think-aloud and constructive interaction in usability testing with children

Constructive interaction provides natural thinking-aloud as test subjects collaborate to solve tasks. Since children may face difficulties in following instructions for a standard think-aloud test, constructive interaction has been suggested as evaluation method when conducting usability testing with children. However, the relationship between think-aloud and constructive interaction is still poorly understood.We present an experiment that compares think-aloud and constructive interaction in usability testing. The experiment involves 60 children with three setups where children apply think-aloud, and constructive interaction in acquainted and non-acquainted pairs. Our results showed only minor significant differences between the setups, but the pairing of the children had impact on identification of usability problems as acquainted dyads identified more problems both in total and of the most severe than non-acquainted dyads and individual testers. Finally, the acquainted pairs reported that they had to put less effort into the testing than the think-aloud and non-acquainted children.

Heuristic evaluation and mobile usability: Bridging the realism gap

Heuristic evaluation (HE) is problematic when applied to mobile technologies, in that contextual influences over use are poorly represented. Here we propose two lightweight variants of HE: the Heuristic Walkthrough (HW) combines HE with scenarios of use, and the Contextual Walkthrough (CW) involves conducting the HW in the field. 11 usability experts were asked to use one of these three approaches to evaluate a mobile device and the usability flaws discovered were compared across technique. HW discovered more critical usability flaws than HE. CW revealed some unique problems relating to I/O and ambient lighting not encountered in the other two approaches. Though contextualizing heuristic evaluation improves the assessment of mobile devices, it appears that it is possible to introduce contextual detail, i.e. to bridge the ’realism gap’, with scenarios rather than expensive in-situ testing.

Mediated intimacy in families: understanding the relation between children and parents

Mediating intimacy between children and their parents is still limited investigated and at the same time, we find that, emerging technologies are about to change and affect the way we interact with each other. In this paper, we report from an empirical study where we investigated the social interaction phenomena that unfold between children and their parents. We used cultural probes and contextual interviews to investigate the intimate acts between children and parents in three families. Our findings show that the intimate act between children and parents share a number of similarities with other types of intimate relations such as strong-tie intimacy (couples cohabiting). However, we also identified several issues of intimacy unique to the special relation between children and their parents. These unique acts of intimacy propose challenges when designing technologies for mediated intimacy in families.

Studying driver attention and behaviour for three configurations of GPS navigation in real traffic driving

Global Positioning System (GPS) navigation systems were amongst the top selling consumer technologies in 2008 and research has indicated that such technologies could affect driving behaviour. In this paper, we study how different output configurations (audio, visual and audio-visual) of a GPS system affect driving behaviour and performance. We conducted field experiments in real traffic with 30 subjects. Our results illustrated that visual output not only causes a substantial amount of eye glances, but also led to a decrease in driving performance. Adding audio output decreased the number of eye glances, but we found no significant effects on driving performance. Although the audio configuration implied much fewer eye glances and improved driving performance, several participants expressed preference for the audio/visual output.

Exploring context-awareness for ubiquitous computing in the healthcare domain

Ubiquitous technologies have potentials to play major roles in different real world organizational settings. One of the areas where applying ubiquitous technologies has been given a lot of attention is in the healthcare domain. Here, users are frequently on the move while at the same time relying increasingly on centralized computerized information. In this paper, we explore ubiquitous technologies in the real world through two studies in the healthcare domain. First, we look at the use and usability of a ubiquitous electronic patient record (EPR) system distributed on desktop and laptop computers throughout a large hospital. Secondly, we present an extension to this ubiquitous computing environment in the form of a context-aware mobile computer terminal prototype. The usability of the mobile EPR prototype was evaluated in both laboratory and field settings. Our results indicate that the usefulness of a ubiquitous computing environment supporting work activities in healthcare can benefit from context-aware mobile information access. However, interaction design for such systems must be carefully thought out and thoroughly evaluated. Also, while the use of mobile and stationary computers complement each other very well, we found that the usefulness of ubiquitous computing environments in healthcare may benefit from additional elements such as situated displays at key locations and on key objects, and from seamless integration between the different devices comprising the system as a whole.

A Longitudinal Study of Usability in Health Care - Does Time Heal?

We report from a longitudinal laboratory-based usability evaluation of a health care information system. The purpose of the study was to inquire into the nature of usability problems experienced by novice and expert users, and to see to what extend usability problems of a health care information system may or may not disappear over time, as the nurses get more familiar with it-if time heals poor design? As our method for studying this, we conducted a longitudinal study with two key studies. A usability evaluation was conducted with novice users when an electronic patient record system was being deployed in a large hospital. After the nurses had used the system in their daily work for 15 months, we repeated the evaluation. Our results show that time does not heal. Although some problems were not experiences as severe, they still remained after 1 year of extensive use. On the basis of our findings, we discuss implications for evaluating usability in health care.