Beryl Plimmer

INTERACTING with sketched interface designs: an evaluation study

Digital hand-drawn sketches provide a new and unique way of interacting with a prototype user interface design while it is still rendered as a sketch. The successful use of prototypes and scenarios for exploring design ideas is well documented. Hand-sketched designs have also been found preferable to formal diagrams during early design. The study reported here shows that interacting with digital sketches adds an exciting new dimension to the interface design process, we found that people do more revisions and more accurate revisions with digital sketches.

Computer-aided sketching to capture preliminary design

This paper describes the vital role of freehand sketching in the design process. When designers first tackle a design problem they usually do so by sketching. We will explore the essential elements of sketching that make it so helpful to problem solving. We then examine how current computer interfaces interfere with the sketching process, and go on to establish the requirements for an environment to support sketching. Finally we describe a system under development to integrate sketching into a visual programming environment (Visual Basic).

Ink features for diagram recognition

The ability to automatically recognize a sketch accurately is important to computer-based diagramming. Many recognition techniques have been proposed but few researchers have reported the use of formal methods to select the most appropriate ink features for recognition algorithms. We have used a statistical approach to identify the most important distinguishing features of ink for dividing text and shapes. We implemented these into an existing recognition engine and conducted a comparative evaluation. Our feature set more successfully classified a range of common diagram elements than two existing dividers.

3D input for 3D worlds

Virtual Worlds present a 3D space to the user. However, input devices are typically 2D. This unnatural mapping reduces the engagement of the experience. We are exploring using Wii controllers to provide 3D gesture-based input to the 3D virtual world, Second Life. By evaluating its usability, we found that gesture-based interfaces are appealing and natural for hand gestures such as wave but difficult to map to facial expressions.

Signing on the tactile line: A multimodal system for teaching handwriting to blind children

We present McSig, a multimodal system for teaching blind children cursive handwriting so that they can create a personal signature. For blind people handwriting is very difficult to learn as it is a near-zero feedback activity that is needed only occasionally, yet in important situations; for example, to make an attractive and repeatable signature for legal contracts. McSig aids the teaching of signatures by translating digital ink from the teachers stylus gestures into three non-visual forms: (1) audio pan and pitch represents the x and y movement of the stylus; (2) kinaesthetic information is provided to the student through a force-feedback haptic pen that mimics the teachers stylus movement; and (3) a physical tactile line on the writing sheet is created by the haptic pen. McSig has been developed over two major iterations of design, usability testing and evaluation. The final step of the first iteration was a short evaluation with eight visually impaired children. The results suggested that McSig had the highest potential benefit for congenitally and totally blind children and also indicated some areas where McSig could be enhanced. The second prototype incorporated significant modifications to the system, improving the audio, tactile and force-feedback. We then ran a detailed, longitudinal evaluation over 14 weeks with three of the congenitally blind children to assess McSigs effectiveness in teaching the creation of signatures. Results demonstrated the effectiveness of McSig—they all made considerable progress in learning to create a recognizable signature. By the end of ten lessons, two of the children could form a complete, repeatable signature unaided, the third could do so with a little verbal prompting. Furthermore, during this project, we have learnt valuable lessons about providing consistent feedback between different communications channels (by manual interactions, haptic device, pen correction) that will be of interest to others developing multimodal systems.

SketchSet: Creating Euler diagrams using pen or mouse

Euler diagrams form the basis of various visual languages but tool support for creating them is generally limited to generic diagram editing software using mouse and keyboard interaction. A more natural and convenient mode of entry is via a sketching interface which facilitates greater cognitive focus on the task of diagram creation. Previous work has developed sketching interfaces for Euler diagrams drawn with ellipses. This paper presents SketchSet, the first sketch tool for Euler diagrams whose curves can be circles, ellipses, or arbitrary shapes. SketchSet allows the creation of formal diagrams via point and click interaction. The user drawn diagram, in sketched or formal format, is automatically converted to a diagram in the other format, thus maintaining both views. We provide a mechanism that allows semantic differences between the sketch and the formal diagram to be rectified automatically. Finally, we present a user study that evaluates the effectiveness of the tool.

The power of automatic feature selection: Rubine on steroids

Digital ink features drive recognition engines. Intuitively, we understand that particular features are of more value for some problems than others. Likewise, inclusion of poor features may be detrimental to recognition success. Many different ink features have been proposed for ink recognition, and most work well for the context that they are employed. However given a new problem it is not clear which of the already defined features will be most useful. We have assembled and categorized a comprehensive feature library and use this with attribute selection algorithms to choose the best features for a specified problem. To verify the effectiveness of this approach the selected features are used to train a Rubines recognizer. We show that a set of complementary features is most effective: poor features adversely affect recognition as do two or more aliases of good features. We have composed a variant of a Rubine recognizer for 3 different datasets and compared these with the Rubines original features, a variant on this InkRubine and

SketchNode: intelligent sketching support and formal diagramming

1. The results show that feature selection can significantly improve recognition rates with this simple algorithm thus verifying our hypothesis that the right combination of features for a problem is one key to recognition success.

RCA: experiences with an IDE annotation tool

The primary motivation for building SketchNode is to provide an environment for exploring how people use tools to create, arrange, edit and interpret graph diagrams. It has two equivalent interfaces: sketching and diagramming, so that the functional requirements and advantages and disadvantages of the differences can be studied. In this paper we describe two iterations of SketchNode, in particular the computational intelligence required to maintain a sketch that appears hand-drawn and the complexity of providing two interfaces that are equivalent in terms of interaction and visualization. The development and usability tests presented here contribute to the understanding of what intelligent sketch diagramming tools can support and the interaction paradigm of dual visualization tools.

Technical Section: Using data mining for digital ink recognition: Dividing text and shapes in sketched diagrams

Ink annotation is a common method for recording feedback on a paper document. However, reviewing code on paper is difficult due to its non-linear nature. This project extends existing research ideas to develop a digital ink annotation tool within an Integrated Development Environment (IDE). The aim is to provide code reviewers with an effective tool for directly commenting on code within the IDE. We describe scenarios where ink annotation would provide benefits, along with requirements and our implementation of the Rich Code Annotation Tool (RCA).