Paul Schmieder

SketchSet: Creating Euler diagrams using pen or mouse

Euler diagrams form the basis of various visual languages but tool support for creating them is generally limited to generic diagram editing software using mouse and keyboard interaction. A more natural and convenient mode of entry is via a sketching interface which facilitates greater cognitive focus on the task of diagram creation. Previous work has developed sketching interfaces for Euler diagrams drawn with ellipses. This paper presents SketchSet, the first sketch tool for Euler diagrams whose curves can be circles, ellipses, or arbitrary shapes. SketchSet allows the creation of formal diagrams via point and click interaction. The user drawn diagram, in sketched or formal format, is automatically converted to a diagram in the other format, thus maintaining both views. We provide a mechanism that allows semantic differences between the sketch and the formal diagram to be rectified automatically. Finally, we present a user study that evaluates the effectiveness of the tool.

Automatic evaluation of sketch recognizers

We present our toolkit to automatically evaluate recognition algorithms. There are few published comparative evaluations of sketch recognition algorithms and those that exist do not provide benchmarking or direct comparisons because standardised data and an evaluation platform is not available. By unifying data collection, labelling and evaluation in one tool, fair, flexible and comprehensive evaluations are possible. Currently we have 6 existing recognizers integrated into this tool. With our initial evaluations of these recognizers we have observed that the context from which training data is taken has an effect on recognition success rates. These results suggest that an evaluation platform such as this is a powerful adjunct for sketch recognition research.

Generating systems from multiple sketched models

Diagrams are often used to model complex systems: in many cases several different types of diagrams are used to model different aspects of the system. These diagrams, perhaps from multiple stakeholders of different specialties, must be combined to achieve a full abstract representation of the system. Many CAD tools offer multi-diagram integration; however, sketch-based diagramming tools are yet to tackle this difficult integration problem. We extend the diagram sketching tool InkKit to combine software engineering sketches of different types. Our extensions support software design processes by providing a sketch-based approach that allows the iterative creation of multiple outputs interacting with one another from the inter-linked models. We demonstrate that InkKit can generate a functional system consisting of a user interface with processes to submit and retrieve data from a database from sketched user interfaces designs and sketched entity relationship diagrams.

Thumbs Up: 3D Gesture Input on Mobile Phones Using the Front Facing Camera

We use the front facing camera in a smart phone to capture gesture input. Thumb gestures performed above the camera are recognized and used to invoke commands. In contrast to other input modalities the camera requires no device movements and no valuable screen space is used. To be viable, this type of interaction requires gestures which are comfortable and memorable for the user and real-time accurate recognition of those gestures. Given the performance constraints of phones and their cameras we needed to determine whether accurate and reliable recognition is possible and identify types of gestures that are recognizable and user appropriate. As a proof of concept, we conducted a user study testing three gestures for performance and user satisfaction. The results demonstrate that the 3D gestural input is successful and we provide detailed insights into successful recognition strategies for this novel interaction modality.

Distortion techniques for sketching interaction

Using sketching as the application domain, this research compares current distortion techniques for enlarging content. The goal is to develop a distortion lens which allows for a natural and uncomplicated drawing and writing experience on an electronic device.