André Berton

SmartKom-Mobile Car: User Interaction with Mobile Services in a Car Environment

People tend to spend an increasing amount of time in their cars and therefore desire high comfort, safety, and efficiency in that environment. A large variety of electronic devices has been made available to meet these requirements in the vehicle. These electronic devices should allow for speech interaction in order to minimize driver distraction and to maximize driver comfort.

Speech Interaction with the Internet -- A User Study

The arrival of smart phones significantly impacts the automotive environment. People tend to use their mobile Internet connection manually while driving which distracts and endangers the drivers safety. In order to reduce driver distraction a speech based interface to Internet services is essential. However, before developing a speech dialog system in a new domain, a data collection from real users is needed. In this report, a web-based user study is conducted, which aims at getting knowledge about how users would interact with Internet services by speech. The user study was separated in a questionnaire and audio recordings based on graphically depicted scenarios the subjects had to solve orally. The results show that the users are willing to use and trust in speech dialog systems. The speaking styles occurring in the audio data were classified into natural, command and keyword style. The occurrence differed depending on the web task category whereas natural speaking style was most frequently used by the subjects.

Towards a flexible UI model for automotive human-machine interaction

In this paper we present an approach for creating user interfaces from abstract representations for the automotive domain. The approach is based on transformations between different user interface abstraction levels. Existing user interface representation methods are presented and evaluated. The impact of specific requirements for automotive human-machine interaction is discussed. Considering these requirements a process based on transformation rules is outlined to allow for flexible integration of external infotainment applications coming from mobile devices or web sources into the in-car interaction environment.

Development of Speech-Based In-Car HMI Concepts for Information Exchange Internet Apps

The permanent use of smartphones impacts the automotive environment. People tend to use their smartphone’s Internet capabilities manually while driving, which endangers the driver’s safety. Therefore, an intuitive in-car speech interface to the Internet is crucial in order to reduce driver distraction. Before developing an in-car speech dialog system to a new domain, you have to examine which speech-based human-machine interface concept is the most intuitive. This work in progress report describes the design of various human-machine interface concepts which include speech as main input and output modality. These concepts are based on two different dialog strategies: a command-based and a conversational speech dialog. Different graphical user interfaces, one including an avatar, have been designed in order to best support the speech dialog strategies and to raise the level of naturalness in the interaction. For each human-machine interface concept a prototype which allows for an online hotel booking has been developed. These prototypes will be evaluated in driving simulator experiments on usability and driving performance.

Evaluation of speech-based HMI concepts for information exchange tasks

HighlightsWe compare different speech-based in-car HMI concepts in a driving simulator study.The HMI concepts are evaluated in terms of usability and driver distraction.The comparison of speech dialog strategies revealed only differences in usability.The use of a GUI impaired the driving performance and raised gaze-based distraction.An avatar does not additionally raise driver distraction but is not accepted by users. Due to the mobile Internet revolution, people tend to browse the Web while driving their car which puts the drivers safety at risk. Therefore, an intuitive and non-distractive in-car speech interface to the Web needs to be developed. Before developing a new speech dialog system (SDS) in a new domain developers have to examine the users preferred interaction style and its influence on driving safety.This paper reports a driving simulation study, which was conducted to compare different speech-based in-car human-machine interface concepts concerning usability and driver distraction. The applied SDS prototypes were developed to perform an online hotel booking by speech while driving. The speech dialog prototypes were based on different speech dialog strategies: a command-based and a conversational dialog. Different graphical user interface (GUI) concepts (one including a human-like avatar) were designed in order to support the respective dialog strategy the most and to evaluate the effect of the GUI on usability and driver distraction.The results show that only few differences concerning speech dialog quality were found when comparing the speech dialog strategies. The command-based dialog was slightly better accepted than the conversational dialog, which seems to be due to the high concept error rate of the conversational dialog. A SDS without GUI also seems to be feasible for the driving environment and was accepted by the users. The comparison of speech dialog strategies did not reveal differences in driver distraction. However, the use of a GUI impaired the driving performance and increased gaze-based distraction. The presence of an avatar was not appreciated by participants and did not affect the dialog performance. Concerning driver distraction, the virtual agent did neither negatively affect the driving performance nor increase visual distraction.The results implicate that in-car SDS developers should take both speaking styles into consideration when designing an SDS for information exchange tasks. Furthermore, developers have to consider reducing the content presented on the screen in order to reduce driver distraction. A human-like avatar was not appreciated by users while driving. Research should further investigate if other kinds of avatars might achieve different results.

Adaptive personalization of multimodal vehicular interfaces using a hybrid recommendation approach with fuzzy preferences

User-adaptive and situation-aware presentation of information on multimodal interfaces is an important research area for coping with the increasing information load on the driver. This paper describes a comprehensive recommendation approach for inferring vague individual preferences under uncertain conditions by using fuzzy preference relations. The approach was applied to rank internet information on a multimodal driver interface in the vehicle. New information gets successively downloaded to the car by using digital media broadcast and the driver can query information (e.g. press review, cheap petrol stations, snow reports) by using natural language queries. Incoming information elements are gradually matched to ontology categories by a fuzzy membership value based on term frequency. User perferences are learned from interaction and recommendations for content items is based on the resulting preference model. Three different types of fuzzy relations are aggregated to an overall preference score: explicit preferences, implicit preferences, and global preferences. The later influence preference values of one driver based on other drivers in vehicular ad-hoc networks. Therefore, the approach is not solely based on the presence of one kind of interaction data. Furthermore, the situation is modeled using the concept of granules and influences the overall preference score.

Comparison of speech-based in-car HMI concepts in a driving simulation study

This paper reports experimental results from a driving simulation study in order to compare different speech-based in-car human-machine interface concepts. The effects of the use of a command-based and a conversational in-car speech dialog system on usability and driver distraction are evaluated. Different graphical user interface concepts have been designed in order to investigate their potential supportive or distracting effects. The results show that only few differences concerning speech dialog quality were found when comparing the speech dialog strategies. The command-based dialog was slightly better accepted than the conversational dialog, which can be attributed to the limited performance of the systems language understanding component. No differences in driver distraction were revealed. Moreover, the study revealed that speech dialog systems without graphical user interface were accepted by participants in the driving environment and that the use of a graphical user interface impaired the driving performance and increased gaze-based distraction. In the driving scenario, the choice of speech dialog strategies does not have a strong influence on usability and no influence on driver distraction. Instead, when designing the graphical user interface of an in-car speech dialog systems, developers should consider reducing the content presented on the display device in order to reduce driver distraction.

In-Car Multi-Domain Spoken Dialogs: A Wizard of Oz Study

Mobile Internet access via smartphones puts demands on in-car infotainment systems, as more and more drivers like to access the Internet while driving. Spoken dialog systems support the user by less distracting interaction than visual/hapticbased dialog systems. To develop an intuitive and usable spoken dialog system, an extensive analysis of the interaction concept is necessary. We conducted a Wizard of Oz study to investigate how users will carry out tasks which involve multiple applications in a speech-only, user-initiative infotainment system while driving. Results show that users are not aware of different applications and use anaphoric expressions in task switches. Speaking styles vary and depend on type of task and dialog state. Users interact efficiently and provide multiple semantic concepts in one utterance. This sets high demands for future spoken dialog systems.

Out-of-Domain Spoken Dialogs in the Car: A WoZ Study

Mobile Internet access via smartphones puts demands on in-car infotainment systems, as more and more drivers like to access the Internet while driving. Spoken dialog systems (SDS) distract drivers less than visual/haptic-based dialog systems. However, in conversational SDSs drivers might speak utterances which are not in the domain of the SDS and thus cannot be understood. In a Wizard of Oz study, we evaluate the effects of out-of-domain utterances on cognitive load, driving performance, and usability. The results show that an SDS which reacts as expected by the driver, is a good approach to control incar infotainment systems, whereas unexpected SDS reactions might cause severe accidents. We evaluate how a dialog initiative switch, which guides the user and enables him to reach his task goal, performs.

Evaluation of In-Car SDS Notification Concepts for Incoming Proactive Events

Due to the mobile Internet revolution, people communicate increasingly via social networks and instant messaging applications using their smartphones. In order to stay “always connected” they even use their smartphone while driving their car which puts the driver safety at risk. In order to reduce driver distraction an intuitive speech interface which provides the driver with proactively incoming events needs to be developed. Before developing a new speech dialog system developers have to examine what the user’s preferred interaction style is. This paper reports from a recent driving simulation study in which several speech-based proactive notification concepts for incoming events in different contextual situations are evaluated. 4 different speech dialog and 2 graphical user interface concepts, one including an avatar, were designed and evaluated on usability and driving performance. The results show that there are significant differences when comparing the speech dialog concepts. Informing the user verbally achieves the best result concerning usability. Earcons are perceived to be the least distractive. The presence of an avatar was not accepted by the participants and led to an impaired steering performance.