Christine Kühnel

I'm home: Defining and evaluating a gesture set for smart-home control

Abstract Mobile phones seem to present the perfect user interface for interacting with smart environments, e.g. smart-home systems, as they are nowadays ubiquitous and equipped with an increasing amount of sensors and interface components, such as multi-touch screens. After giving an overview on related work this paper presents the adapted design methodology proposed by Wobbrock et al. (2009) for the development of a gesture-based user interface to a smart-home system. The findings for the new domain, device and gesture space are presented and compared to findings by Wobbrock et al. (2009) . Three additional steps are described: A small pre-test survey, a mapping and a memory test and a performance test of the implemented system. This paper shows the adaptability of the approach described by Wobbrock et al. (2009) for three-dimensional gestures in the smart-home domain. Elicited gestures are described and a first implementation of a user interface based on these gestures is presented.

Evaluating multimodal systems: a comparison of established questionnaires and interaction parameters

This paper describes the analysis of established and new questionnaires concerning their applicability for the assessment of quality aspects of multimodal systems. To this purpose, an experiment with 27 participants interacting with a a smart-home system via a voice interface, a smartphone-based interface and a multimodal interface, was conducted. Interaction parameters were assessed and related to constructs measured with these questionnaires. The results indicate that some of the questionnaires are suitable for evaluating multimodal interfaces. On the basis of correlations with interaction parameters subscales of these questionnaires can be mapped to quality aspects, such as effectiveness and efficiency. Recommendations are given how to meet two important evaluation requirements, namely which questionnaire to use for comparing two or more systems or system versions and how to identify factors or components in a system that have to be improved. This is another step forward to establish evaluation methods for multimodal systems.

Evaluating talking heads for smart home systems

In this paper we report the results of a user study evaluating talking heads in the smart home domain. Three noncommercial talking head components are linked to two freely available speech synthesis systems, resulting in six different combinations. The influence of head and voice components on overall quality is analyzed as well as the correlation between them. Three different ways to assess overall quality are presented. It is shown that these three are consistent in their results. Another important result is that in this design speech and visual quality are independent of each other. Furthermore, a linear combination of both quality aspects models overall quality of talking heads to a good degree.

Quantifying Quality Aspects of a Multimodal Interactive System

After discussing the quantification of multimodal interaction, input quality and output quality, in this chapter the results are consolidated to give an idea of the full picture. Based on the taxonomy introduced in Chap. 2 an attempt is made to quantify the quality aspects of an entire multimodal interactive system. To this end the chapter will first give a short introduction by taking up a few concepts described in Chap. 2. The research question is motivated shortly, the methodology described and results illustrated. After discussing these in the light of findings presented in the previous chapters conclusions are drawn and a short summary is given. The content of this chapter has been partially published in Kuhnel et al. (2011a).

Quality of Experiencing Multi-Modal Interaction

In this chapter, we discuss the contributions of different modalities to the overall quality of multi-modal interaction. After reviewing some common systematics and findings concerning multi-modality, we present experimental results from several multi-modal scenarios, involving different (human-to-human and human-to-machine) interaction paradigms, different degrees of interactivity, and different (speech, audio, video, touch, gesture) modalities.The results show that the impact of each modality on overall quality in interaction depends heavily onthe scenario and degree of interactivity. Complementary modalities are not considered in this paper, but the models presented allow predicting overall system quality on the basis of individual modality ratings with an appropriate accuracy. These models still have to be validated in order to be used as tools for system developers estimating whether adding modalities will have an impact on the quality experienced by the user.

Evaluating Multimodal Systems

As seen in the previous chapter, multimodal systems are well-established—at least in the research community studying HCI. But during the development process and—at the latest–once the system is built, methods for quantitative assessment are needed. In fact, as systems are usually meant to fullfill certain needs of, assist or even replace the human, the user’s perspective needs to be considered from the very start of conceiving a system. This approach of taking into account the potential user group and its characteristics has culminated in the user-centered or participatory design approach (Schuler and Namioka 1993), namely involving the user in the design process. Some accepted methods for user-centered design, such as the cognitive walkthrough, are discussed below, please refer to Vredenburg et al. (2002) for a survey of user-centered design practice.

Quantifying Output Quality Aspects

In Chap. 3.4 it has been stated that ‘often, multimodal systems are multimodal in input only, at best presenting multimedia output’. At the same time there is a rising interest in real multimodal output. While there are several possible combinations of output modalities, in this work, two types are discussed in detail: output via a talking head (see Chap. 3.4.1) and the combination of the talking head with graphics.

The Multimodal Interactive System: INSPIRE_Me

As a precondition for the work presented a multimodal system is needed to enable the realization of extended studies to address the research questions motivated in Chap. 1. It has become apparent in the previous chapters that quite a number of multimodal systems have been designed, mostly for a similar purpose and only recently for consumers. The nature of the studies conducted did not allow the use of an off-the-shelf product, as these systems usually only permit so-called black box for an explanation of black box and glass box evaluations. Thus, an existing system accessible at the home institution was extended. In this chapter the domain as well as the different input and output modalities chosen are motivated in the light of existing research, state of the art and the research questions addressed. Furthermore, the different versions of the multimodal system used for the underlying studies are described.

Quantifying Multimodal Interaction Behaviour

After having laid the theoretical and practical foundations—in terms of the multimodal system INSPIRE_Me described in Chap. 3—it is now time to address the research questions that are the core of this book.