Orit Shaer

Reality-based interaction: a framework for post-WIMP interfaces

We are in the midst of an explosion of emerging human-computer interaction techniques that redefine our understanding of both computers and interaction. We propose the notion of Reality-Based Interaction (RBI) as a unifying concept that ties together a large subset of these emerging interaction styles. Based on this concept of RBI, we provide a framework that can be used to understand, compare, and relate current paths of recent HCI research as well as to analyze specific interaction designs. We believe that viewing interaction through the lens of RBI provides insights for design and uncovers gaps or opportunities for future research.

The TAC paradigm: specifying tangible user interfaces

This paper introduces a paradigm for describing and specifying tangible user interfaces (TUIs). The proposed Token and Constraints (TAC) paradigm captures the core components of TUIs while addressing many of the conceptual challenges unique to building these interfaces. The paradigm enables the description of a broad range of TUIs by providing a common set of constructs. Thus, the TAC paradigm lays the foundation for a high-level description language and a software toolkit for TUIs. We evaluate the proposed paradigm by testing its ability to specify a wide variety of existing TUIs.

A specification paradigm for the design and implementation of tangible user interfaces

Tangible interaction shows promise to significantly enhance computer-mediated support for activities such as learning, problem solving, and design. However, tangible user interfaces are currently considered challenging to design and build. Designers and developers of these interfaces encounter several conceptual, methodological, and technical difficulties. Among others, these challenges include: the lack of appropriate interaction abstractions, the shortcomings of current user interface software tools to address continuous and parallel interactions, as well as the excessive effort required to integrate novel input and output technologies. To address these challenges, we propose a specification paradigm for designing and implementing Tangible User Interfaces (TUIs), that enables TUI developers to specify the structure and behavior of a tangible user interface using high-level constructs which abstract away implementation details. An important benefit of this approach, which is based on User Interface Description Language (UIDL) research, is that these specifications could be automatically or semi-automatically converted into concrete TUI implementations. In addition, such specifications could serve as a common ground for investigating both design and implementation concerns by TUI developers from different disciplines. Thus, the primary contribution of this article is a high-level UIDL that provides developers from different disciplines means for effectively specifying, discussing, and programming a broad range of tangible user interfaces. There are three distinct elements to this contribution: a visual specification technique that is based on Statecharts and Petri nets, an XML-compliant language that extends this visual specification technique, as well as a proof-of-concept prototype of a Tangible User Interface Management System (TUIMS) that semi-automatically translates high-level specifications into a program controlling specific target technologies.

Reality-based interaction: unifying the new generation of interaction styles

We are in the midst of an explosion of emerging human-computer interaction techniques that have redefined our understanding of both computers and interaction. We propose the notion of Reality-Based Interaction (RBI) as a unifying concept that ties together a large subset of these emerging interaction styles. Through RBI we are attempting to provide a framework that can be used to understand, compare, and relate current paths of HCI research. Viewing interaction through the lens of RBI can provide insights for designers and allows us to find gaps or opportunities for future development. Furthermore, we are using RBI to develop new evaluation techniques for features of emerging interfaces that are currently unquantifiable.

G-nome surfer: a tabletop interface for collaborative exploration of genomic data

Molecular and computational biologists develop new insights by gathering heterogeneous data from genomic databases and leveraging bioinformatics tools. Through a qualitative study with 17 participants, we found that molecular and computational biologists experience difficulties interpreting, comparing, annotating, sharing, and relating this vast amount of biological information. We further observed that such interactions are critical for forming new scientific hypotheses. These observations motivated the creation of G-nome Surfer, a tabletop interface for collaborative exploration of genomic data that implements multi-touch and tangible interaction techniques. G-nome Surfer was developed in close collaboration with domain scientists and is aimed at lowering the threshold for using bioinformatics tools. A first-use study with 16 participants found that G-nome Surfer enables users to gain biological insights that are based on multiple forms of evidence with minimal overhead.

The tangible video editor: collaborative video editing with active tokens

In this paper we introduce the Tangible Video Editor (TVE), a multi-user, tangible interface for sequencing digital video. We present a new approach to tabletop interaction by using multiple handheld computers embedded in plastic tokens. Drawing from the rich physical experience of tradition film editing techniques we designed the TVE to engage multiple users in a collaborative process and encourage the exploration of narrative ideas. We used active tokens to provide a malleable interface, enabling users to organize the interface components in unspecified ways. Our implementation improves upon common projection-based tabletop interfaces in a number of ways including a design for use beyond dedicated two dimensional spaces and a naturally scaling screen resolution.

Smart Blocks: a tangible mathematical manipulative

We created Smart Blocks, an augmented mathematical manipulative that allows users to explore the concepts of volume and surface area of 3-dimensional (3D) objects. This interface supports physical manipulation for exploring spatial relationships and it provides continuous feedback for reinforcing learning. By leveraging the benefits of physicality with the advantages of digital information, this tangible interface provides an engaging environment for learning about surface area and volume of 3D objects.

The design, development, and deployment of a tabletop interface for collaborative exploration of genomic data

In this paper, we reflect on the design, development, and deployment of G-nome Surfer; a multi-touch tabletop user interface for collaborative exploration of genomic data. G-nome Surfer lowers the threshold for using advanced bioinformatics tools, reduces the mental workload associated with manipulating genomic information, and fosters effective collaboration. We describe our two-year-long effort from design strategy to iterations of design, development, and evaluation. This paper presents four main contributions: (1) a set of design requirements for supporting collaborative exploration in data-intensive domains, (2) the design, implementation, and validation of a multi-touch tabletop interface for collaborative exploration, (3) a methodology for evaluating the strengths and limitations of tabletop interaction for collaborative exploration, and (4) empirical evidence for the feasibility and value of integrating tabletop interaction in college-level education.

A collaborative environment for engaging novices in scientific inquiry

We describe the design, implementation, and evaluation of GreenTouch, a collaborative environment that enables novice users to engage in authentic scientific inquiry. GreenTouch consists of a mobile user interface for capturing data in the field, a web application for data curation in the cloud, and a tabletop interface for exploratory analysis of heterogeneous data. This paper contributes: 1) the design, implementation, and validation of a collaborative environment which allows novices to engage in scientific data capture, curation, and analysis; 2) empirical evidence for the feasibility and value of integrating interactive surfaces in college-level education based on an in situ study with 54 undergraduate students; and 3) insights collected through iterative design, providing concrete lessons and guidelines for designing multi-touch interfaces for collaborative inquiry of complex domains.

SynFlo: A Tangible Museum Exhibit for Exploring Bio-Design

We present SynFlo, a tangible museum exhibit for exploring bio-design. SynFlo utilizes active and concrete tangible tokens to allow visitors to experience a playful biodesign activity through complex interactivity with digital biological creations. We developed two versions of SynFlo: one that combines active tokens with real concrete objects (i.e. labware) and one that consists of only abstract active tokens. Results from an evaluation in a museum indicate that both systems support learning. We discuss design choices for biology education tools to overcome confounders of biology and facilitate positive engagement and learning.