Kent Wittenburg

Grammar-based articulation for multimedia document design

This paper describes an approach to the problem of articulating multimedia information based on parsing and syntax-directed translation that uses Relational Grammars. This translation is followed by a constraint-solving mechanism to create the final layout. Grammatical rules provide the mechanism for mapping from a representation of the content and context of a presentation to forms that specify the media objects to be realized. These realization forms include sets of spatial and temporal constraints between elements of the presentation. Individual grammars encapsulate the “look and feel” of a presentation and can be used as generators of such a style. By making the grammars sensitive to the requirements of the output medium, parsing can introduce flexibility into the information realization process.

A comparison between spoken queries and menu-based interfaces for in-car digital music selection

Distracted driving is a significant issue for our society today, and yet information technologies, including growing digital music collections, continue to be introduced into the automobile. This paper describes work concerning methods designed to lessen cognitive load and distracting visual demands on drivers as they go about the task of searching for and listening to digital music. The existing commercial paradigms for retrieval—graphical or spoken menu traversal, and text-based search—are unsatisfactory when cognitive resources are limited and keyboards are unavailable. We have previously proposed to use error-tolerant spoken queries [26] combined with direct modalities such as buttons mounted on the steering wheel [7]. In this paper, we present in detail the results of an experiment designed to compare the industry standard approach of hierarchical graphical menus to our approach. We found our proposed interface to be more efficient and less distracting in a simulated driving task.

Multi-projectors and implicit interaction in persuasive public displays

Recent advances in computer video projection open up new possibilities for real-time interactive, persuasive displays. Now a display can continuously adapt to a viewer so as to maximize its effectiveness. However, by the very nature of persuasion, these displays must be both immersive and subtle. We have been working on technologies that support this application including multi-projector and implicit interaction techniques. These technologies have been used to create a series of interactive persuasive displays that are described.

Multi-user multi-touch games on diamondtouch with the DTFlash toolkit

Games and other forms of tabletop electronic entertainment are a natural application of the new multi-user multi-touch tabletop technology DiamondTouch [3]. Electronic versions of familiar tabletop games such as ping-pong or air hockey require simultaneous touch events that can be uniquely associated with different users. Multi-touch two-handed gestures useful for, e.g., rotating, stretching, capturing, or releasing also have natural applications for entertainment applications built on electronic tabletops. Here we show a set of games that are illustrative of the capabilities of an underlying authoring toolkit we call DTFlash. DTFlash is designed so that those familiar with Macromedia Flash authoring tools can add multi-user multi-touch gestures and behaviors to web-enabled games and other applications for the DiamondTouch table.

A speech-in list-out approach to spoken user interfaces

Spoken user interfaces are conventionally either dialogue-based or menu-based. In this paper we propose a third approach, in which the task of invoking responses from the system is treated as one of retrieval from the set of all possible responses. Unlike conventional spoken user interfaces that return a unique response to the user, the proposed interface returns a shortlist of possible responses, from which the user must make the final selection. We refer to such interfaces as Speech-In List-Out or SILO interfaces. Experiments show that SILO interfaces can be very effective, are highly robust to degraded speech recognition performance, and can impose significantly lower cognitive load on the user as compared to menu-based interfaces.

Relational grammars: theory and practice in a visual language interface for process modeling

Relational Grammars (RGs) are one of the higher-dimensional grammar formalisms that have been proposed for representing visual languages (VLs). This paper serves as an overview of the formalism as well as a case study of its application in a visual language interface (VLI) for process modeling. Relational Grammars are a member of the context-free family of Constraint Multiset Grammars (Marriott and Meyer, this volume), and several subclasses of Relational Grammars have been proposed that have been motivated by the algorithmic demands of specific applications. The applications that have been explored go beyond the exercise of recognizing visual expressions that can be constructed with standard graphical editors. Here we focus on a business process modeling tool called ShowBiz and discuss how grammatical representation and parsing is used dynamically in creating aggregations for process encapsulation and visual focusing.

Visualizing set-valued attributes in parallel with equal-height histograms

Visualization of set-valued attributes in multi-dimensional information visualization systems remains a relatively unexplored problem. Here we introduce a novel method for visualizing set-valued attributes that we call the singleton set distribution view and integrate it into an interactive multi-dimensional attribute visualization tool utilizing parallel bargrams (aka equal-height histograms) as its main visual motif. We discuss our design rationale and report on the results of an evaluation study.

Visual Language Parsing: If I Had a Hammer...

Since the 1960s, grammatical formalisms and parsing methods developed originally for natural language strings have been extended to represent and process two-dimensional visual expressions such as mathematics notation and various kinds of diagrams. But despite all of the effort, there has been negligible impact on human-computer interfaces to support visual modes of communication. Why? As with all tech transfer issues, some of the reasons may be beyond a researchers control. However, I believe that two of the contributing factors in the case of visual language (VL) parsing can and should be addressed by the research field. First, the field needs to consolidate and communicate its results. This is in fact not trivial for higher-dimensional visual language representation and parsing, and I will try to illustrate why. Second, researchers have to look harder for the right application domains. One of the obvious applications is the interpretation of visual language expressions constructed with GUIs. While grammatical representation and parsing may bring something to the table, the problem of interpretation may be solvable with simpler techniques. I will discuss some other application areas and my experience with them: design support, smart screen layout for electronic publishing, and visual focusing for attributed graphs.

Setting the bar for set-valued attributes

In this paper, we present an interactive visualization method for set-valued attributes that maintains the advantages of item-oriented views and interactions found in parallel multivariate visualizations such as bargrams (equal-height histograms). The challenge is to accommodate renderings of an item when it appears multiple times in set-valued attribute views while at the same time preserving value- and item-based selection, brushing, and filtering. Such techniques can help users derive particular types of insights into data based on distributions and correlations of attribute values.

Envisioning grid vulnerabilities: multi-dimensional visualization for electrical grid planning

Electrical grid planning aims at optimizing the grid through the control of the performance and placement of electrical assets in order to minimize failures or vulnerabilities. With this purpose, grid planners carry out an initial stage of data exploration using a large volume of incident and equipment data collected over extensive time periods. In current practice these tasks are performed manually, which makes it very difficult to recognize patterns and gain insights into the data. In this paper, we propose a parallel multivariate visualization technique as a suitable approach for improving the existing practice. Based on the usage of an interactive visualization tool called BarExam, we demonstrate the feasibility of this visualization technique for displaying the dataset and present example insights that this visualization technique can provide to grid planners.