Gerrit C. van der Veer

Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes

In multi-agent, multi-user environments, users as well as agents should have a means of establishing who is talking to whom. In this paper, we present an experiment aimed at evaluating whether gaze directional cues of users could be used for this purpose. Using an eye tracker, we measured subject gaze at the faces of conversational partners during four-person conversations. Results indicate that when someone is listening or speaking to individuals, there is indeed a high probability that the person looked at is the person listened (p=88%) or spoken to (p=77%). We conclude that gaze is an excellent predictor of conversational attention in multiparty conversations. As such, it may form a reliable source of input for conversational systems that need to establish whom the user is speaking or listening to. We implemented our findings in FRED, a multi-agent conversational system that uses eye input to gauge which agent the user is listening or speaking to.

GTA: Groupware task analysis Modeling complexity

The task analysis methods discussed in this presentation stem from Human-Computer Interaction (HCI) and Ethnography (as applied for the design of Computer Supported Cooperative Work CSCW), different disciplines that often are considered conflicting approaches when applied to the same design problems. Both approaches have their strength and weakness, and an integration of them does add value to the early stages of design of cooperation technology. In order to develop an integrated method for groupware task analysis (GTA) a conceptual framework is presented that allows a systematic perspective on complex work phenomena. The framework features a triple focus, considering (a) people, (b) work, and (c) the situation. Integrating various task-modeling approaches requires vehicles for making design information explicit, for which an object oriented formalism will be suggested. GTA consists of a method and framework that have been developed during practical design exercises. Examples from some of these cases will illustrate our approach.

Task based groupware design: putting theory into practice

Designing Groupware systems requires methods and tools that cover all aspects of Groupware systems. We present a method that utilizes known theoretical insights and makes them usable in practice. In our method, the design of Groupware systems is driven by an extensive task analysis followed by structured design and iterative evaluation using usability criteria. Using a combination of multiple complementary representations and techniques, a wide range of aspects of Groupware design is covered. The method is built on our experiences and is used in practice by several companies and educational institutes in Europe. We define the design process, the models needed and the tools that support the design process.

On the interaction between system and user characteristics

Abstract Abstract. The development of a representational framework, based on Morans Command Language Grammar, for constructing adaptive computer systems is the main topic of this paper. A model of the user interface is developed of which two aspects are discussed: the mental model in the mind of the user and the perceptual model on the side of the machine. Cognitive style and personality factors are considered in their relation to the development of the mental model. The concept of metacommunication is analysed and discussed on all levels of Morans CLG.

Personalized Digital Television

This chapter presents the recommendation techniques applied in Personal Program Guide (PPG). This is a system generating personalized Electronic Program Guides for Digital TV. The PPGmanages a user model that stores the estimates of the individual user’s preferences for TV program categories. This model results from the integration of di¡erent preference acquisitionmodules that handle explicit user preferences, stereotypical information about TV viewers, and information about the user’s viewing behavior. The observation of the individual viewing behavior is particularly easy because the PPG runs on the set-top box and is deeply integrated with the TV playing and the video recording services o¡ered by that type of device.

Catching the eye: management of joint attention in cooperative work

In this paper, we show how different elements of awareness information in groupware systems can be defined in terms of conveying attentive states of the participants. Different kinds of awareness are distinguished: at macro- and micro-level, the latter consisting of workspace awareness and conversational awareness. We summarize the functional elements of micro-level awareness, organizing them hierarchically in terms of their relation to the attention of participants. We further discuss how groupware systems can capture and represent awareness by means of attention-based metaphors, and give an example of a virtual meeting room in which the gaze direction of the participants is conveyed by means of modern ‘imaging’ eyetracking technology.

Individual differences and the user interface

Individual differences in cognitive functions are relevant in introducing novioes to computing. The interaction between the user and the computer may be improved if these psychological characteristics are taken into account in the design of the user interface and in the design of introductory courses. In.section 1 of this paper an analysis is made of individual differences and of the possibilities of adaptation. The user interface is described as an opportunity for adaptation. Methods of adaptation are systematically related to the changeability of individual differences. Illustrated by an overview of previous experimental work, some principles of adaptation are demonstrated in relation to individual differences on variables that are relevant in human-computer interaction. Prior knowledge of mathematics has been shown to be related to success and failure in learning a programming language, and to motivation for using computers. In section 2 ways to prevent the course drop-out rate by structuring the teach...Individual differences in cognitive functions are relevant in introducing novioes to computing. The interaction between the user and the computer may be improved if these psychological characteristics are taken into account in the design of the user interface and in the design of introductory courses. In.section 1 of this paper an analysis is made of individual differences and of the possibilities of adaptation. The user interface is described as an opportunity for adaptation. Methods of adaptation are systematically related to the changeability of individual differences. Illustrated by an overview of previous experimental work, some principles of adaptation are demonstrated in relation to individual differences on variables that are relevant in human-computer interaction. Prior knowledge of mathematics has been shown to be related to success and failure in learning a programming language, and to motivation for using computers. In section 2 ways to prevent the course drop-out rate by structuring the teaching method are illustrated. Strategies of information processing in the user interface, in relation to metacommunication and teaching, are described in section 3 as another example of an individual difference variable that may be attended to in teaching, resulting in reduction of problem solving time. Some cognitive styles are related to success in programming. The choice of adequate educational examples may improve programming habits, as illustrated in section 4. Style of representation and spatial ability are relevant for the development of mental models of computer system. In section 5 some field studies on teaching novices to work with office systems illustrate these relations. Teaching strategies, educational examples, and metaphors, may neutralize the handicap some novices have because of these variables.

Collaborative Practices that Support Creativity in Design

Design is a ubiquitous, collaborative and highly material activity. Because of the embodied nature of the design profession, designers apply certain collaborative practices to enhance creativity in their everyday work. Within the domain of industrial design, we studied two educational design departments over a period of eight months. Using examples from our fieldwork, we develop our results around three broad themes related to collaborative practices that support the creativity of design professionals: 1) externalization, 2) use of physical space, and 3) use of bodies. We believe that these themes of collaborative practices could provide new insights into designing technologies for supporting a varied set of design activities. We describe two conceptual collaborative systems derived from the results of our study.

Why conversational agents should catch the eye

We studied whether the gaze direction of users indicates whom they are speaking or listening to in multiparty conversations. Results show when someone is listening or speaking to individuals, there is indeed a high probability that the person looked at is the person listened (p=88%) or spoken to (p=77%). We implemented these findings in a multi-agent conversational system that uses eye input to gauge which agent the user is listening or speaking to.

Towards a Multimodal Interaction Space: categorisation and applications

Based on many experiences of developing interactive systems by the authors, a framework for the description and analysis of interaction has been developed. The dimensions of this multimodal interaction space have been identified as sensory modalities, modes and levels of interaction. To illustrate and validate this framework, development of multimodal interaction styles is carried out and interactions in the real world are studied, going from theory to practice and back again. The paper describes the framework and two recent projects, one in the field of interactive architecture and another in the field of multimodal HCI research. Both projects use multiple modalities for interaction, particularly movement based interaction styles.