Nils Dahlbäck

Wizard of Oz studies: why and how

We discuss current approaches to the development of natural-language dialo=we systems, and claim that they do not sufficiently consider the unique qualities of man-machine interaction as distinct from general human discourse. We conclude that empirical studies of this unique comnnmication situation is required for the development of user-friendly interactive systems. One way of achieving this is through the use of so-called Wizard of Oz studies. We describe our work in tlis area. The focus is on the practical execution of the studies and the methodological conclusions that we have drawn on the basis of our experience. While the focus is on natural language interfams, the methods used and the conclusions drawn from the results obtained m of relevance also to other kinds of intelligent interfaces.

A glass box approach to adaptive hypermedia

Utilising adaptive interface techniques in the design of systems introduces certain risks. An adaptive interface is not static, but will actively adapt to the perceived needs of the user. Unless carefully designed, these changes may lead to an unpredictable, obscure and uncontrollable interface. Therefore the design of adaptive interfaces must ensure that users can inspect the adaptivity mechanisms, and control their results. One way to do this is to rely on the users understanding of the application and the domain, and relate the adaptivity mechanisms to domain-specific concepts. We present an example of an adaptive hypertext help system POP, which is being built according to these principles, and discuss the design considerations and empirical findings that lead to this design.

Empirical studies of discourse representations for natural language interfaces

We present the results from a series of experiments aimed at uncovering the discourse structure of man-machine communication in natural language (Wizard of Oz experiments). The results suggest the existence of different classes of dialogue situations, requiring computational discourse representations of various complexity. Important factors seem to be the number of different permissible tasks in the system and to what extent the system takes initiative in the dialogue. We also analyse indexical expressions and especially the use of pronouns, and suggest a psychological explanation of their restricted occurrence in these types of dialogues.

A case study of factor influencing role improvisation in crisis response teams

Common characteristics of crisis situations are ambiguous and unplanned for events. The need for improvised roles can therefore be an imperative factor for the success of an operation. The aim of this study is to deepen the understanding of the processes taking place during improvised work “as it happens”. A case study of a crisis management team at work is presented and provides an in-depth analysis of the information and communication flow of persons acting in improvised roles, including contextual factors influencing the task at hand. The analysis suggests that three main factors lay behind decreased performance by the team when some of its members were forced to take on roles for which they lacked professional training; lack of language skills, lack of domain knowledge and insufficient organizational structure of the tasks. Based on the observations from this case study, we suggest three ways of improving a team’s performance and hence resilience when forced to improvise due to lack of personnel in one or more required competence areas. These are training to take on the responsibility for tasks or roles outside ones professional area of specialization, developing formal routines for changes in roles and tasks and developing and using tools and routines for information sharing.

Distilling dialogues - A method using natural dialogue corpora for dialogue systems development

We report on a method for utilising corpora collected in natural settings. It is based on distilling (re-writing) natural dialogues to elicit the type of dialogue that would occur if one the dialogue participants was a computer instead of a human. The method is a complement to other means such as Wizard of Oz-studies and un-distilled natural dialogues. We present the distilling method and guidelines for distillation. We also illustrate how the method affects a corpus of dialogues and discuss the pros and cons of three approaches in different phases of dialogue systems development.

Towards a Dialogue Taxonomy

Two interrelated points are made in this paper. First, that some of the characteristics of the language used in spoken dialogues are also observed in typed dialogues, since they are a reflection of the fact that it is a dialogue, rather than the fact that it is spoken. A corollary of this is that some of the results obtained in previous work on typed dialogue, especially Wizard of Oz simulations of human-computer natural language dialogues, can be of use to workers on spoken language dialogue systems. Second, it is claimed that we need a more fine grained taxonomy of different kinds of dialogues. Both for making it possible to make use of results obtained by other workers in developing dialogue systems, and for the development of our theoretical understanding of the influence of non-linguistic factors on the language used in human-computer dialogues. A number of such dimensions are described and their influence on the language used is illustrated by results from a empirical studies of language use.

3D Visualization as a Communicative Aid in Pharmaceutical Advice-Giving over Distance

Background Medication misuse results in considerable problems for both patient and society. It is a complex problem with many contributing factors, including timely access to product information. Objective To investigate the value of 3-dimensional (3D) visualization paired with video conferencing as a tool for pharmaceutical advice over distance in terms of accessibility and ease of use for the advice seeker. Methods We created a Web-based communication service called AssistancePlus that allows an advisor to demonstrate the physical handling of a complex pharmaceutical product to an advice seeker with the aid of 3D visualization and audio/video conferencing. AssistancePlus was tested in 2 separate user studies performed in a usability lab, under realistic settings and emulating a real usage situation. In the first study, 10 pharmacy students were assisted by 2 advisors from the Swedish National Co-operation of Pharmacies’ call centre on the use of an asthma inhaler. The student-advisor interview sessions were filmed on video to qualitatively explore their experience of giving and receiving advice with the aid of 3D visualization. In the second study, 3 advisors from the same call centre instructed 23 participants recruited from the general public on the use of 2 products: (1) an insulin injection pen, and (2) a growth hormone injection syringe. First, participants received advice on one product in an audio-recorded telephone call and for the other product in a video-recorded AssistancePlus session (product order balanced). In conjunction with the AssistancePlus session, participants answered a questionnaire regarding accessibility, perceived expressiveness, and general usefulness of 3D visualization for advice-giving over distance compared with the telephone and were given a short interview focusing on their experience of the 3D features. Results In both studies, participants found the AssistancePlus service helpful in providing clear and exact instructions. In the second study, directly comparing AssistancePlus and the telephone, AssistancePlus was judged positively for ease of communication (P = .001), personal contact (P = .001), explanatory power (P < .001), and efficiency (P < .001). Participants in both studies said that they would welcome this type of service as an alternative to the telephone and to face-to-face interaction when a physical meeting is not possible or not convenient. However, although AssistancePlus was considered as easy to use as the telephone, they would choose AssistancePlus over the telephone only when the complexity of the question demanded the higher level of expressiveness it offers. For simpler questions, a simpler service was preferred. Conclusions 3D visualization paired with video conferencing can be useful for advice-giving over distance, specifically for issues that require a higher level of communicative expressiveness than the telephone can offer. 3D-supported advice-giving can increase the range of issues that can be handled over distance and thus improve access to product information.

The shared priorities measure as a way of assessing team strategic awareness: a bridge between self-assessment and the deep blue sea of field recordings

Objective, easy to use, easy to comprehend, high face-validity assessment methods for measuring shared awareness in teams are hard to find. This paper describes an experiment where a new measure called Shared Priorities, which is based on ranking of self-generated strategic items, is tested. Trained teams were compared to non-trained teams in a dynamic problem-solving task in terms of performance and shared awareness. The shared priorities measure was used alongside other, well-documented measures of team awareness based on self-rating. The results show that the Shared Priorities measure correlate with performance and could also distinguish between trained and non-trained teams. However, the Shared Priorities measure did not correlate with the other team measures, suggesting that it captures a different quality of team work than the self-rating measures. Further, the shared priorities measure was found to be easily administered and gained a high user acceptance.

Statistical modelling of team training in a microworld study

A command and control environment is a dynamic and complex setting with complicated technical systems where teams of operators interact to reach shared goals. This study presents an experiment in which we, by means of Structural Equation Modeling (SEM), explain the relations between basic concepts of command and control environments: mental workload, frustration, situational awareness, and performance. This paper reports a LISREL analysis of the Baroutsi, Berggren, Nählinder, & Johansson (2013) data. From that data, a new latent variable “Frustration” emerges, which now can be included in the model.

Robustness in speech based interfaces: sharing the tricks of the trade

Speech is as innate to humans as breathing. We speak to our cats, our dogs and some of us even speak to our plants. While speech is a very natural form of communication for people, its adoption rate has been slow in major commercial computer systems. Part of this may be due to the fact that insufficient usability expertise has gone into the design of many speech systems; many having been created by speech developers instead of HCI experts. Given that speech is not 100% accurate, nor will be in the foreseeable future, a special set of skills and knowledge must be applied to the design of speech interfaces to make them usable. Part of the goal of the workshop is to expand the base of this knowledge through sharing from experience.