Michael Haller

Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces

Many research projects have demonstrated the benefits of bimanual interaction for a variety of tasks. When choosing bimanual input, system designers must select the input device that each hand will control. In this paper, we argue for the use of pen and touch two-handed input, and describe an experiment in which users were faster and committed fewer errors using pen and touch input in comparison to using either touch and touch or pen and pen input while performing a representative bimanual task. We present design principles and an application in which we applied our design rationale toward the creation of a learnable set of bimanual, pen and touch input commands.

The NiCE Discussion Room: Integrating Paper and Digital Media to Support Co-Located Group Meetings

Current technological solutions that enable content creation and sharing during group discussion meetings are often cumbersome to use, and are commonly abandoned for traditional paper-based tools, which provide flexibility in supporting a wide range of working styles and task activities that may occur in a given meeting. Paper-based tools, however, have their own drawbacks; paper-based content is difficult to modify or replicate. We introduce a novel digital meeting room design, the NiCE Discussion Room, which integrates digital and paper tools into a cohesive system with an intuitive pen-based interface. The combination of digital and paper media provides groups with a flexible design solution that enables them to create, access, and share information and media from a variety of sources to facilitate group discussions. This paper describes the design solution, along with results from a user study conducted to evaluate the usability and utility of the system.

A real-time shadow approach for an augmented reality application using shadow volumes

Shadows add a level of realism to a rendered image. Moreover, they are used as visual clues to determine spacial relationships and real-time shadows will gain importance in current real-time computer graphics applications for this reason. Twenty-five years ago, Crow published the shadow volume approach for determining shadowed regions of a scene. In this paper we present a modified real-time shadow volume algorithm that can be used in an Augmented/Mixed Reality application. Finally, the proposed concepts provide a novel sense of visual output of an AR application.

Occlusion-aware menu design for digital tabletops

In this paper, we describe the design of menus for multi-user digital tabletops. On direct input surfaces, occlusions created by the users hand decrease interaction performance with menus. The key design criteria are to avoid these occlusions and to adapt the menu placement to the users handedness and position on the tabletop. We present an adaptive menu placement method based on direct touch and pen tracking that allows correct menu placement around the table. As an extension, we propose adding a gesture input area for fast interaction which can be partly occluded by the users hand.

Shared design space: sketching ideas using digital pens and a large augmented tabletop setup

Collaborative Augmented Reality (AR) setups are becoming increasingly popular. We have developed a collaborative tabletop environment that is designed for brainstorming and discussion meetings. Using a digital pen, participants can annotate not only virtual paper, but also real printouts. By integrating both forms of physical and digital paper, we combine virtual and real 2d drawings, and digital data which are overlaid into a single information space. In this paper, we describe why we have integrated these devices together in a unique way and how they can be used efficiently during a design process.

CRISTAL: a collaborative home media and device controller based on a multi-touch display

While most homes are inherently social places, existing devices designed to control consumer electronics typically only support single user interaction. Further, as the number of consumer electronics in modern homes increases, people are often forced to switch between many controllers to interact with these devices. To simplify interaction with these devices and to enable more collaborative forms of device control, we propose an integrated remote control system, called CRISTAL (Control of Remotely Interfaced Systems using Touch-based Actions in Living spaces). CRISTAL enables people to control a wide variety of digital devices from a centralized, interactive tabletop system that provides an intuitive, gesture-based interface that enables multiple users to control home media devices through a virtually augmented video image of the surrounding environment. A preliminary user study of the CRISTAL system is presented, along with a discussion of future research directions.

Improving Menu Interaction for Cluttered Tabletop Setups with User-Drawn Path Menus

Japanese calligraphy is the art of brush writing where a person writes Japanese characters with a Chinese brush against a sheet of paper. We have implemented a mechanism to capture the process of producing Japanese calligraphy using MERLs DiamondTouch (DT) table. We add a very thin metal wire along the length of the brush to carry an electric signal from the writers body through the brush tuft and ink to the table. As the brush tuft is touches a sheet of paper placed on the surface of the DT table, the ink in the tuft carries the signal from the users to DT. We capture the movement of the brush tuft to produce the visual and auditory representations of the writing process and for later replay.Many digital tabletop systems have a graphical user interface (GUI) that features context (or pop-up) menus. While linear and pie menus are commonly used for direct pen and touch interaction, their appearance can be problematic on a digital tabletop display, where physical objects might occlude menu items. We propose a user-drawn path menu, that appears along a custom path to avoid such occlusions. This paper introduces four different metaphors for user-drawn context menus: the Fan Out Menu, the Card Deck Menu, the Pearl String Menu, and the Trail Menu. It also presents the results we acquired from a user study, where participants were able to work faster when using our user-drawn menus, on cluttered tabletop setups.

FlexSense: a transparent self-sensing deformable surface

We present FlexSense, a new thin-film, transparent sensing surface based on printed piezoelectric sensors, which can reconstruct complex deformations without the need for any external sensing, such as cameras. FlexSense provides a fully self-contained setup which improves mobility and is not affected from occlusions. Using only a sparse set of sensors, printed on the periphery of the surface substrate, we devise two new algorithms to fully reconstruct the complex deformations of the sheet, using only these sparse sensor measurements. An evaluation shows that both proposed algorithms are capable of reconstructing complex deformations accurately. We demonstrate how FlexSense can be used for a variety of 2.5D interactions, including as a transparent cover for tablets where bending can be performed alongside touch to enable magic lens style effects, layered input, and mode switching, as well as the ability to use our device as a high degree-of-freedom input controller for gaming and beyond.

An Adaptable Rear-Projection Screen Using Digital Pens And Hand Gestures

INTOI is a rear-projection setup which combines accurate pen tracking with hand gesture recognition. The hardware consists of an Anoto pattern printed on a special rear-projection foil and an infrared tracking system. INTOI is a low-cost system that is scalable and provides highly accurate input (to less than 1mm). Finally, our setup supports a novel multi-user interaction that combines simultaneous interaction of both hand and pen gesture input.

IncreTable, a mixed reality tabletop game experience

IncreTable is a mixed reality tabletop game inspired by The Incredible Machine. Users can combine real and virtual game pieces in order to solve puzzles in the game. Game actions include placing virtual domino blocks with digital pens, controlling a virtual car by modifying the virtual terrain through a depth camera interface or controlling real robots to topple over real and virtual dominoes.