Ellen Yi-Luen Do

Ambiguous intentions: a paper-like interface for creative design

Interfaces for conceptual and creative design should recognize and interpret drawings. They should also capture users’ intended ambiguity, vagueness, and imprecision and convey these qualities visually and through interactive behavior. Freehand drawing can provide this information and it is a natural input mode for design. We describe a pen-based interface that acquires information about ambiguity and precision from freehand input, represents it internally, and echoes it to users visually and through constraint based edit behavior.

Computational Support for Sketching in Design: A Review

Computational support for sketching is an exciting research area at the intersection of design research, human–computer interaction, and artificial intelligence. Despite the prevalence of software tools, most designers begin their work with physical sketches. Modern computational tools largely treat design as a linear process beginning with a specific problem and ending with a specific solution. Sketch-based design tools offer another approach that may fit design practice better. This review surveys literature related to such tools. First, we describe the practical basis of sketching — why people sketch, what significance it has in design and problem solving, and the cognitive activities it supports. Second, we survey computational support for sketching, including methods for performing sketch recognition and managing ambiguity, techniques for modeling recognizable elements, and human–computer interaction techniques for working with sketches. Last, we propose challenges and opportunities for future advances in this field.

Intentions in and relations among design drawings

Abstract Designers use drawings to explore alternatives and to test ideas. We report here on two studies on design and drawing. The first study of design drawing symbols aims to determine whether and to what extent it is possible to infer, interpret, or even guess what a designer was thinking about by looking at the drawings she has made. In the second study we examined a collection of drawings for the design of a house to investigate the systems of design transformations. Drawings are characterized by drawing style, projection type, and key elements. We analyzed the relationships among the drawings and developed a notation system for documenting these relationships.

Annotating and sketching on 3D web models

This paper reports on our progress and findings in building a Web annotation system for non-immersive 3D virtual environments. Over the last two years, we developed and tested two systems for collaborating designers to comment on virtual 3D models. Our first system, Redliner [12] lets design team members browse and leave text annotations on surfaces in three-dimensional models. Experience with Redliner, including two user evaluations in different settings, led us to develop Space Pen [13], a second annotation system with improved interaction capabilities. It goes beyond the post-it note metaphor, allowing users to draw in and on the virtual environment.

Demonstrating the electronic cocktail napkin: a paper-like interface for early design

We demonstrate the Electronic Cocktail Napkin, a pen based interface for conceptual design. The project goal is to support design by recognizing, interpreting, and managing drawings, and to serve as an interface for knowledge-based critiquing, simulation, and information retrieval. We demonstrate the Napkin’s facilities for end-user programmable recognition and interpretation, drawing management, and multi-user collaboration. We show applications of the Napkin: (1) indexing visual databases and (2) a front end to a local area network design program.

Don't mind me touching my wrist: a case study of interacting with on-body technology in public

Wearable technology, specifically e-textiles, offers the potential for interacting with electronic devices in a whole new manner. However, some may find the operation of a system that employs non-traditional on-body interactions uncomfortable to perform in a public setting, impacting how readily a new form of mobile technology may be received. Thus, it is important for interaction designers to take into consideration the implications of on-body gesture interactions when designing wearable interfaces. In this study, we explore the third-party perceptions of a users interactions with a wearable e-textile interface. This two-prong evaluation examines the societal perceptions of a user interacting with the textile interface at different on-body locations, as well as the observers attitudes toward on-body controller placement. We performed the study in the United States and South Korea to gain cultural insights into the perceptions of on-body technology usage.

Navigational blocks: navigating information space with tangible media

The Navigational Blocks project demonstrates a tangible user interface that facilitates retrieval of historical stories in a tourist spot. Orientation, movement, and relative positions of physical Blocks support visitor navigation and exploration in a virtual gallery. The Navigational Blocks system provides a physical embodiment of digital information through tactile manipulation and haptic feedback. The simple cubic form of the Blocks is easy to understand and therefore easy to use to manipulate complex digital information. Electromagnets embedded in the Blocks and wireless communication encourage users to quickly rearrange the Blocks to form different database queries.

Drawing on the Back of an Envelope: a framework for interacting with application programs by freehand drawing

Abstract The Back of an Envelope project demonstrates how a calligraphic interface (one that employs a pen or stylus to input freehand drawing marks) can be used in a wide variety of domains, from databases to simulation programs, to 3D modeling, from mostly symbolic diagrams to freeform sketches. The wide variety of drawing types and domains calls for a diverse range of approaches. We describe some of the functionality of our systems, including contextual recognition of symbols and configurations and emergent shape recognition, and some of the calligraphic interfaces we’ve built.

PianoTouch: A wearable haptic piano instruction system for passive learning of piano skills

We present PianoTouch, a wearable, wireless haptic piano instruction system, composed of (1) five small vibration motors, one for each finger, fitted inside a glove, (2) a Bluetooth module mounted on the glove, and (3) piano music output from a laptop. Users hear the piano music and feel the vibrations indicating which finger is used to play the note. We investigate the systems potential for passive learning, i.e. learning piano playing automatically while engaged in everyday activities. In a preliminary study, four subjects learned two songs initially and then wore the PianoTouch glove for 30 minutes while listening to the songs repeated. One of the songs included tactile sensations and the other did not. The study found that after 30 minutes, the PianoTouch subjects were able to play the song accompanied by tactile sensations better than the non-tactile song. These results suggest the value of a more detailed study.

Posey: instrumenting a poseable hub and strut construction toy

We describe Posey, a computationally-enhanced hub-and-strut construction kit for learning and play. Posey employs a ball and socket connection that allows users to move the parts of an assembled model. Hubs and struts are optocoupled through the ball and socket joints using infrared LEDs and photosensors. Wireless transmitters in the hubs send connection and geometry information to a host computer. The host computer assembles a representation of the physical model as the user creates and configures it. Application programs can then use this representation to control computational models in particular domains.