Tomáš Macek

A DOM-based MVC multi-modal e-business

Modality: A particular type physical interface that can be perceived or interacted with by the user (e.g. voice interface, GUI display with keypad etc...) Multi-modal Browser: A browser that enables the user to interact with an application through different modes of intercation (e.g. typically: Voice and GUI). Accordingly a multi-modal-browser provides different moadlities for input and output Ideally it lets the user select at any time the modality that is the most appropriate to perform a particular interaction given this interaction and the users situation Thesis: By improving the user interface, we believe that multi-modal browsing will significantly accelerate the acceptance and growth of m-Commerce. Multiple access mechanisms One interaction mode per device PC Standardized rich visual interface Not suitable for mobile use I need a direct flight from New York to San Francisco after 7:30pm today There are five direct flights from New Yorks LaGuardia airport to San Francisco after 7:30pm today: Delta flight nnn...

Mostly passive information delivery in a car

In this study we present and analyze a mostly passive infotainment approach to presenting information in a car. The passive style is similar to radio listening but content is generated on the fly and it is based on a mixture of personal information (calendar, emails) and public data (news, POI, jokes). The spoken part of the audio is machine synthesized. We explore two modes of operation. The first one is passive only. The second one is more interactive and speech commands are used to personalize the information mix and to request particular information items. Usability and distraction tests were conducted with both systems implemented using the Wizard of Oz technique. Both systems were assessed using multiple objective and subjective metrics and the results indicate that driver distraction was low for both systems. The users differed in the amount of interaction they preferred. Some users preferred more command-driven styles while others were happy with passive presentation. Most of the users were satisfied with the quality of synthesized speech and found it sufficient for the given purpose. In addition, feedback was collected from the subjects on what kind of information they liked listening to and how they would have preferred to ask for specific types of information.

Impact of word error rate on driving performance while dictating short texts

This paper describes the impact of speech recognition word error rate (WER) on drivers distraction in the context of short message dictation. A multi-modal dictation and error correction system was used in a simulated driving environment (Lane Change Test, LCT) to dictate text messages with prescribed semantic content. Driving accuracy was measured using several objective statistics produced by the LCT simulator. We report results for three datasets: 28 LCT trips by native US-English speakers at 40km/h, 23 more trips at 60km/h which had noise added in order to artificially increase WER levels and 22 LCT trips at 60km/h performed by non-native accented speakers. For the two datasets that used 60km/h we observed a moderate correlation between the drivers WER and driving performance statistics such as the mean deviation from ideal track (MDev) and the standard deviation of lateral position (SDLP). This correlation reached statistical significance for all of these statistics in the native dataset, and was significant for the overall SDLP in the non-native dataset. Additionally, we observed that higher WER levels lead to significantly lower message throughput and to significantly lower quality of sent messages, esp. for non-native speakers.

Long Text Reading in a Car

We present here the results of a study focused on text reading in a car. The purpose of this work is to explore how machine synthesized reading is perceived by users. Are the users willing to tolerate deficiencies of machine synthesized speech and trade it off for more current content? What is the impact of listening to it on driver’s distraction? How do the answers to the questions above differ for various types of text content? Those are the questions we try to answer in the presented study. We conducted the study with 12 participants, each facing three types of tasks. The tasks differed in the length and structure of the presented text. Reading out a fable represented an unstructured pleasure reading text. The news represented more structured short texts. Browsing a car manual was an example of working with structured text where the user looks for particular information without much focusing on surrounding content. The results indicate relatively good user acceptance for the presented tasks. Distraction of the driver was related to the amount of interaction with the system. Users opted for controlling the system by buttons on the steering wheel and made little use of the system’s display.

In-car dictation and driver's distraction: a case study

We describe a prototype dictation UI for use in cars and evaluate it by measuring (1) drivers distraction, (2) task completion time, and (3) task completion quality. We use a simulated lane change test (LCT) to assess driving quality while using the prototype, while texting using a cell phone and when just driving. The prototype was used in two modes - with and without a display (eyes-free). Several statistics were collected from the reference and distracted driving LCT trips for a group of 11 test subjects. These statistics include drivers mean deviation from ideal path, the standard deviation of drivers lateral position on the road, reaction times and the amount and quality of entered text. We confirm that driving performance was significantly better when using a speech enabled UI compared to texting using a cell phone. Interestingly, we measured a significant improvement in driving quality when the same dictation prototype was used in eyes-free mode.

Mostly Passive Information Delivery -- a Prototype

In this paper we introduce a new UI paradigm that mimics radio broadcast along with a prototype called Radio One. The approach aims to present useful information from multiple domains to mobile users (e.g. drivers on the go or cell phone users). The information is served in an entertaining manner in a mostly passive style – without the user having to ask for it– as in real radio broadcast. The content is generated on the fly by a machine and integrates a mix of personal (calendar, emails) and publicly available but customized information (news, weather, POIs). Most of the spoken audio output is machine synthesized. The implemented prototype permits passive listening as well as interaction using voice commands or buttons. Initial feedback gathered while testing the prototype while driving indicates good acceptance of the system and relatively low distraction levels.

Recipes for building voice search UIs for automotive

In this paper we describe a set of techniques we found suitable for building multi-modal search applications for automotive environments. As these applications often search across different topical domains, such as maps, weather or Wikipedia, we discuss the problem of switching focus between different domains. Also, we propose techniques useful for minimizing the response time of the search system in mobile environment. We evaluate some of the proposed techniques by means of usability tests with 10 novice test subjects who drove a simulated lane change test on a driving simulator. We report results describing the induced driving distraction and user acceptance.

Interactive Car Owner's Manual User Study

We present results of a user study with a prototype of an interactive speech-enabled car owners user manual assistant. Its purpose is to help the driver learn about various car features and related procedures. The study focused on two scenarios -- when parked and while driving. We also used the Leap Motion gesture recognizer as an alternative to buttons. During the experiment we collected both objective driving data and subjective feedback. Results indicate that the users preferred the electronic user manual to the paper form, although they proposed numerous improvements. One particular concern was discoverability of content. The acceptance of Leap Motion gestures was low when driving, possibly impacted by short time allowed for practicing. Drivers distraction caused by interacting with the multimodal user manual was similar to that of receiving and reading text messages.

Djinn: Interaction Framework for Home Environment Using Speech and Vision

In this paper we describe an interaction framework that uses speech recognition and computer vision to model new generation of interfaces in the residential environment. We outline the blueprints of the architecture and describe the main building blocks. We show a concrete prototype platform where this novel architecture has been deployed and will be tested at the user field trials. EC co-funds this work as part of HomeTalk IST-2001-33507 project.

Speech-based text correction patterns in noisy environment

We present a study focused on observation of methods of dictation and error correction between humans in a noisy environment. The purpose of this study is to gain insight to natural communication patterns which can then be applied to human --- machine interaction. We asked 10 subjects to conduct the standard Lane Change Test (LCT) while dictating messages to a human counterpart who had to note down the message texts. Both parties were located in separate rooms and communicated over Skype. Both were exposed to varying types and levels of noise, which made their communication difficult and forced the subjects to deal with misunderstandings. Dictation of both short and longer messages was tested. We observed how the subjects behaved and we analyzed their communication patterns. We identified and described more then 20 elementary observations related to communication techniques such as synchronization and grounding of parties, error checking and error correction. We also report frequencies of use for each communication pattern and provide basic characteristics of driving distraction during the test.