Kengo Omura

Memory cues for meeting video retrieval

We advocate a new approach to meeting video retrieval based on the use of memory cues. First we present a new survey involving 519 people in which we investigate the types of items people use to review meeting contents (e.g., minutes, video, etc.). Then we present a novel memory study involving 15 subjects in which we investigate what people remember about past meetings (e.g., seating position, etc). Based on these studies and related research we propose a novel framework for meeting video retrieval based on memory cues. Our proposed system graphically represents important memory retrieval cues such as room layout, participants faces and sitting positions, etc.. Queries are formulated dynamically: as the user graphically manipulates the cues, the query results are shown. Our system (1) helps users easily express the <i>cues</i> they recall about a particular meeting, and (2) helps users <i>remember</i> new cues for meeting video retrieval. Finally, we present our approach to automatic indexing of meeting videos, present experiments, and discuss research issues in automatic indexing for retrieval using memory cues.

Affective Meeting Video Analysis

In this paper, we examine the affective content of meeting videos. First, we asked five subjects to manually label three meeting videos using continuous response measurement (continuous-scale labeling in real-time) for energy and valence, (the two dimensions of the human affect space). Then we automatically extracted audio-visual features to characterize the affective content of the videos. We compare the results of manual labeling and low-level automatic audiovisual feature extraction. Our analysis yields promising results which suggest that affective meeting video analysis can lead to very interesting observations useful for automatic indexing

Effects of paper on page turning: comparison of paper and electronic media in reading documents with endnotes

This study compares the performances of paper and electronic media during a reading task that includes frequent page turning. In the experiment, 18 subjects read multi-page documents aloud while referring to endnotes using paper, a large display, and a small display. Results revealed that reading from paper was 6.8% faster than reading from a large electronic display and 11.4% faster than reading from a small electronic display. No difference was found between scores of recognition tests of important words of documents among the three conditions, which indicates that paper is the most effective medium for people to read text speedily without reducing comprehension. Detailed analyses of the reading process show that, in the Paper condition, people perform both text reading and page-turning simultaneously. However, when using computer displays, reading and turning pages were divided completely and performed separately.

Do tablets really support discussion?: comparison between paper, tablet, and laptop PC used as discussion tools

Touch-based tablet devices are starting to be used frequently in meetings and places of discussion. However, are tablets really ideal as discussion tools? Or do they actually obstruct communication? To answer these questions, this paper quantitatively compares discussion processes involving paper, an iPad2, and a laptop PC. We performed an experiment where 12 groups of two participants each (24 participants in total) worked collaboratively by referring to documents and using paper, an iPad2, or a laptop PC as presentation media. We observed verbal and non-verbal interaction between participants. First, we investigated the total amount of speech between two participants and found they spoke more when using paper than when using the electronic media. Next, we observed that participants used more demonstrative pronouns when using paper than when using the iPad2 but used more demonstrative pronouns when using the iPad2 than when using the laptop PC. Also, they made more eye contact when using paper than when using the other media. These results suggest that tablets may not currently be the best media to use when ideas should be actively exchanged, sensitivity is required toward other participants in the discussion, or work progress needs to be shared.

Comparison of paper and computer displays in reading including frequent movement between pages

This paper describes experiments to compare reading with paper versus electronic media when reading with frequently moving back and forth between pages. In the first experiment, eighteen participants read aloud multi-page documents with endnotes in three conditions: paper, a large display, and a small display. Results revealed that reading from paper was 6.8% faster than reading from a large computer display and 11.4% faster than reading from a small computer display. Detailed analyses of the reading process showed that participants performed both reading and page-turning simultaneously in the paper condition. However, when using computer displays, reading and turning pages were performed separately. In the second experiment, 12 participants read documents using electronic system with more effective features for page-turning such as an overview of pages and links from reference symbols to notes. However, paper remained the most efficient medium to support reading with endnotes. This indicates that electronic media should be improved to support between-page navigation.

A visuospatial memory cue system for meeting video retrieval

We present a system based on a new, memory-cue paradigm for retrieving meeting video scenes. The system graically represents important memory retrieval cues such as room layout, participants faces and sitting positions, etc.. Queries are formulated dynamically: as the user graically manipulates the cues, the query results are shown. Our system (1) helps users easily express the <i>cues</i> they recall about a particular meeting; (2) helps users <i>remember</i> new cues for meeting video retrieval. We discuss the experiments that motivate this new approach, implementation, and future work.

Page Navigation on Paper Books and Electronic Media in Reading to Answer Questions

This paper describes experiments comparing reading performance such as reading speed and the accuracy of tasks in reading to answer questions using different media: paper books, a desktop PC, and a tablet PC. As typical examples of reading of this kind, we considered two scenarios: searching answers from text manuals and searching given photographs from photo books. In two experiments conducted according to the two scenarios, participants performed tasks most quickly using a paper book or a PC. They had trouble flipping through pages with a tablet PC. Although the paper book was inferior to the PC in terms of turning over pages continuously with rapidity, participants performed flexible navigation with the paper book such as starting the navigation from various positions of the book and jumping to the target position straight away without hesitating skipping over the target. Based on these results, we provide suggestions to improve a page navigation feature in electronic media.

Docking window framework: supporting multitasking by docking windows

When performing tasks using computers, multiple documents are used with multiple applications. During working with computers, multiple tasks, perhaps involving multiple documents, are switched. This paper presents the Docking Window Framework: an extended multi-window system supporting such multitasking situations. It enables construction of workspaces comprising multiple windows with simple switching of workspaces. Although previous systems emphasized the support of task-switching after workspace construction, the proposed system characteristically supports construction of workspaces through a docking user interface. It also supports operation of multiple windows simultaneously, provides a tile layout of windows to reduce the overhead of window operations, and supports saving and restoration of workspaces. We conducted two experiments to evaluate the system. In window arrangement tasks, participants performed tasks faster using the proposed system than when using a popular window system (Windows XP). Moreover, in task-switching tasks, participants using our system performed multiple tasks in parallel more efficiently.

Optimal Orientation of Text Documents for Reading and Writing

To facilitate interaction with documents in interactive tabletop applications, we explore how the orientation of documents supports efficient reading and writing. In this paper, the orientation of paper documents is systematically investigated in five different experiments. In the first three experiments, 36 Japanese participants, 18 left- and 18 right-handed, explored preferred document orientation for reading horizontal Japanese texts. Experiments 1 and 2 showed that the participants preferred to rotate text documents clockwise if left-handed and counter-clockwise if right-handed relative to their median line. On average, they rotated text documents more when annotating (about 10°) than when reading (about 5°). The difference in document rotation can be explained by the fact that text documents are rotated to facilitate hand manipulation, because they are rotated more when finer hand motions are required. No difference in document angle was found between Japanese horizontal texts and English texts (Experiment 3). In the fourth experiment, we found that right-handed people read text documents faster when they were rotated between –10° and 20° counter-clockwise. Finally, the last experiment explored the effect of document orientation on the performance of transcribing texts. The participants transcribed texts faster when documents were rotated 10° counter-clockwise. In light of the results from these paper-based experiments, we give guidelines for how to present text documents in tabletop applications depending on tasks, documents, and users.

Proposal and Evaluation of a Document Reader that Supports Pointing and Finger Bookmarking

Pointing and finger bookmarking effectively support reading from paper. However, current electronic media do not support these operations. Readers are discouraged to trace or point text with fingers on popular touchscreen tablet devices because the gestures may cause undesired view changes. Also, bookmarking with the current interface does not provide the ease of finger bookmarking. For solving the problems mentioned above, we proposed a document reader that provides seamless switching between pointing/tracing and touch operations, and integrate features that simulate finger bookmarking by using simple gestures. The results of two experiments (proof reading and cross reference reading between pages) show that participants performed the tasks faster with the proposed system than with the conventional touchscreen systems.