Jared N. Bott

The Wiimote and Beyond: Spatially Convenient Devices for 3D User Interfaces

The Nintendo Wii Remote (Wiimote) has served as an input device in 3D user interfaces (3DUIs) but differs from the general-purpose input hardware typically found in research labs and commercial applications. Despite this, no one has systematically evaluated the device in terms of what it offers 3DUI designers. Experience with the Wiimote indicates that its an imperfect harbinger of a new class of spatially convenient devices, classified in terms of spatial data, functionality, and commodity design. This tutorial presents techniques for using the Wiimote in 3DUIs. It discusses the devices strengths and how to compensate for its limitations, with implications for future spatially convenient devices.

Exploring 3D gestural interfaces for music creation in video games

In recent years the popularity of music and rhythm-based games has experienced tremendous growth. However almost all of these games require custom hardware to be used as input devices, and these devices control only one or two similar instruments. In this paper we describe One Man Band, a prototype video game for musical expression that uses novel 3D spatial interaction techniques using accelerometer-based motion controllers. One Man Band provides users with 3D gestural interfaces to control both the timing and sound of the music played, with both single and collaborative player modes. We further investigate the ability to detect different musical gestures without explicit selection of mode, giving the user the ability to seamlessly transition between instrument types with a single input device. A formal user study is then presented comparing the musical interface of One Man Band to that of Nintendos Wii Music. Our results indicate that users generally preferred the interface of One Man Band over that of Wii Music. We also found that users desire to express their own ideas and have explicit control of the melodies created in music-based video games.

A pen-based tool for visualizing vector mathematics

We present VectorPad, a novel, pen-based application for three-dimensional vector mathematics visualization. VectorPad allows users to define vectors and perform mathematical operations on them through the recognition of handwritten mathematics. The user interface consists of a sketching area, where the user can write vector definitions and operations such as addition, subtraction, scalar multiplication, and cross product, and a 3D graph for visualization. Vectors are visualized dynamically on the graph, which can be manipulated by the user. We also performed a short, informal user study evaluating the user interface and visualizations of VectorPad. Results from the study show that visualizations were generally well liked but the application needs to provide a more comprehensive set of visualization tools as well as refinement to some of the animations.

Now or later: an initial exploration into user perception of mathematical expression recognition feedback

Mathematical handwriting recognition is an important method of mathematics input for computers. While strides in recognition have been made in recent years, recognition is still at a level where mistakes are common and often inexplicable from the users point-of-view. As a result, recognition mistakes can cause user distraction and frustration. We examine how user preference for real-time or batch recognition mode is affected by recognition accuracy and the number of expressions entered. Our results show that users prefer real-time recognition when working with multiple expressions; however, with high accuracy recognition, users did not prefer one recognition mode over the other.

Plugging in and into code bubbles: the code bubbles architecture

Code Bubbles is an attempt to redefine the user interface for an integrated programming environment. As it represents a whole new user interface, implementing it as a plug‐in is inherently difficult. We get around this difficulty by combining two different plug‐in architectures, a standard one based on registrations and callbacks and a message‐based one that puts the plug‐in at arms length and defines a narrower two‐way interface. This paper describes both how we have implemented Code Bubbles as a plug‐in to Eclipse and how Code Bubbles itself is implemented as a set of plug‐ins representing the different aspects of the environment, using both traditional and message‐based plug‐in architectures as appropriate. It also shows how the resultant architecture is flexible enough to support collaboration, different back ends, and a cloud‐based environment. Copyright

Code bubbles: a practical working-set programming environment

Our original work on the Code Bubbles environment demonstrated that a working-set based framework for software development showed promise. We have spent the past several years extending the underlying concepts into a fully-functional system. In our demonstration, we will show the current Code Bubbles environment for Java, how it works, how it can be used, and why we prefer it over more traditional programming environments. We will also show how we have extended the framework to enhance software development tasks such as complex debugging, testing, and collaboration. This paper describes the features we will demonstrate.

One Man Band: A 3D Gestural Interface for Collaborative Music Creation

In recent years the popularity of music and rhythm-based games has experienced tremendous growth. However, almost all of these games require custom hardware to be used as input devices, and these devices control only one or two similar instruments. In this paper we describe One Man Band, a prototype video game for musical expression that uses novel 3D spatial interaction techniques using accelerometer-based motion controllers. One Man Band provides users with 3D gestural interfaces to control both the timing and sound of the music played, with both single and collaborative player modes. We further investigate the ability to detect different musical gestures without explicit selection of mode, giving the user the ability to seamlessly transition between instrument types with a single input device.

Exploring Multi-touch Contact Size for Z-Axis Movement in 3D Environments

In this paper we examine two methods for using relative contact size as an interaction technique for 3D environments on multi-touch capacitive touch screens. We refer to interpreting relative contact size changes as “pressure” simulation. We conducted a 2 x 2 within subjects experimental design using two methods for pressure estimation (calibrated and comparative) and two different 3D tasks (bidirectional and unidirectional). Calibrated pressure estimation was based upon a calibration session, whereas comparative pressure estimation was based upon the contact size of each initial touch. The bidirectional task was guiding a ball through a hoop, while the unidirectional task involved using pressure to rotate a stove knob. Results indicate that the preferred and best performing pressure estimation technique was dependent on the 3D task. For the bidirectional task, calibrated pressure performed significantly better, while the comparative method performed better for the unidirectional task. We discuss the implications and future research directions based on our findings.

The Influence of Multi-Touch Interaction on Procedural Training

This paper explores the use of multi-touch interaction in a 3D training environment as a way to enhance learning of sensorimotor skills as well as procedural knowledge. We present a between subjects experiment with 36 participants distributed into 3 groups that use multi-touch interaction, interaction with the physical apparatus, and a control group using basic mouse-based interaction. A post-training test carried out 3 days later evaluated performance in conducting the real world task from memory. Results show that the multi-touch interaction and the real world groups had significantly better performance scores than the mouse interaction group, with no significant difference between multi-touch and real world groups. Our results demonstrate that multi-touch interaction trained participants on the task as well as training on the actual equipment, indicating multi-touch interaction is a potential replacement for the physical apparatus when doing procedural training.

The WOZ Recognizer: A Wizard of Oz Sketch Recognition System

Sketch recognition has the potential to be an important input method for computers in the coming years, particularly for STEM (science, technology, engineering, and math) education. However, designing and building an accurate and sophisticated sketch recognition system is a time-consuming and daunting task. Since sketch recognition mistakes are still common, it is important to understand how users perceive and tolerate recognition errors and other user interface elements with these imperfect systems. In order to solve this problem, we developed a Wizard of Oz sketch recognition tool, the WOZ Recognizer, that supports controlled recognition accuracy, multiple recognition modes, and multiple sketching domains for performing controlled experiments. We present the design of the WOZ Recognizer and our process for representing recognition domains using graphs and symbol alphabets. In addition, we discuss how sketches are altered, how to control the WOZ Recognizer, and how users interact with it. Finally, we present an expert user case study that examines the WOZ Recognizer’s usability.