Mathias Frisch

Diagram editing on interactive displays using multi-touch and pen gestures

Creating and editing graphs and node-link diagrams by means of digital tools are crucial activities in domains such as software or business process modeling. However, these tools have several drawbacks with regard to interaction techniques and usability. In order to address these issues, we investigate the promising combination of pen and multi-touch input on interactive displays. In this work, we contribute a gesture set to make the interaction with diagrams more efficient and effective by means of pen and hand gestures. Thereby, two prevalent mental models are supported: structural editing and sketching. The gesture set is based on the results of a previous pilot study asking users for suggestions to accomplish diagram editing tasks on tabletops. In this paper, we provide a careful analysis of the resulting user-elicited gestures. We propose solutions to resolve ambiguities within this gesture collection and discuss design decisions for a comprehensible diagram editor. We also present the multi-touch and pen gesture set as a fully implemented prototype for diagram editing on interactive surfaces.

Grids & guides: multi-touch layout and alignment tools

Precise alignment of graphical objects and the creation of accurate layouts are crucial activities in many applications, such as graphics design tools, presentation software or graph editors. Surface computing is very promising for these application domains but not fully explored yet. In this paper we contribute two tools which support layout tasks on interactive displays: interactive grids and multi-touch alignment guides. Both tools allow the precise positioning of graphical objects in a flexible and fluent way by multi-touch input. Direct bimanual interaction and physical metaphors are applied to arrange objects along straight lines and curves. A formative user evaluation showed promising results with regard to a productive and easy use of the tools.

Neat: a set of flexible tools and gestures for layout tasks on interactive displays

Creating accurate layouts of graphical objects is an important activity in many graphics applications, such as design tools, presentation software or diagram editors. In this paper, we are contributing Natural and Effective Layout Techniques (Neat). The system provides a consistent set of multi-touch tools and gestures for aligning and distributing graphical objects on interactive surfaces. NEAT explicitly considers expert requirements and supports a rich and consistent set of layout functions. Amongst others, it minimizes visual distraction by layout tools, combines separate steps of interaction to compound ones and allows effective interaction by combining multi-touch and pen input. Furthermore, Neat provides a set of bimanual gestures for achieving layout tasks in a quick and effective way without explicitly invoking any tools. From initial expert user feedback we derive several principles for layout tools on interactive displays.

Towards seamless semantic zooming techniques for UML diagrams

Models become increasingly important for software development processes. Though there is a multitude of software modeling tools available, the handling of complex UML diagrams is still difficult. In particular, the visualization of a global overview and of logical interrelationships between certain elements arising from refinements of diagrams can be improved. We address these problems and propose the usage of semantic zooming with different levels of detail and describe intuitive interaction techniques to ease the navigation between different diagrams in software models.

Off-screen visualization techniques for class diagrams

Visual representations of node-link diagrams are very important for the software development process. In many situations large diagrams - probably consisting of hundreds of nodes and edges - have to be edited and explored. In state-of-the-art modeling tools these activities are often accompanied by time consuming panning and zooming. In this paper we contribute the application of off-screen visualization techniques to the domain of node-link diagrams in general and to UML class diagrams in particular. The basic idea of the approach is to give a contextual view of all nodes which are clipped from the current viewport. Nodes are represented by proxy elements located within an interactive border region. The proxies show information of the associated off-screen nodes and can be used to quickly navigate to the respective node. However, there are several challenges when this technique is adapted to node-link diagrams, for example concerning the change of edge routing or scalability. We describe the design space of this approach and present different visualization and interaction techniques in detail. Furthermore, we conducted a formative evaluation of our first prototype. Based on the observations made during the evaluation, we came to final suggestions how particular techniques should be combined.

View infinity: a zoomable interface for feature-oriented software development

Software product line engineering provides efficient means to develop variable software. To support program comprehension of software product lines (SPLs), we developed View Infinity, a tool that provides seamless and semantic zooming of different abstraction layers of an SPL. First results of a qualitative study with experienced SPL developers are promising and indicate that View Infinity is useful and intuitive to use.

Visual Support for Understanding Product Lines

The C preprocessor is often used in practice to implement variability in software product lines. Using #ifdef statements provokes problems such as obfuscated source code, yet they will still be used in practice at least in the medium-term future. With CIDE, we demonstrate a tool to improve understanding and maintaining code that contains #ifdef statements by visualizing them with colors and providing different views on the code.

T4 - transparent and translucent tangibles on tabletops

In many cases, Tangible User Interfaces allow the manipulation of digital content with physical objects recognized by an interactive tabletop. Usually, such tangible objects are made of opaque wood or synthetic materials, thereby occluding the display. In this paper, we systematically investigate the promising potential of tangibles entirely made of transparent or translucent materials. Besides visualizing content directly below a manipulable tangible, transparent objects also facilitate direct touch interaction with the content below, dynamic illumination and glowing effects. We propose a comprehensive design space for transparent tangibles on tabletops based on a thorough review of existing work. By reporting on our own experiments and prototypes, we address several gaps in this design space, regarding aspects of both interaction and visualization. These include the illumination of tangibles as well as the precise input with transparent tangibles for which we also present the promising results of an initial user study. Finally, benefits and shortcomings of transparent tangibles are discussed and resulting design considerations are presented.

Enhancing UML sketch tools with digital pens and paper

Drawing diagrams is one of the most important activities during software development processes. A multitude of sketch-based diagram tools were developed to enhance sketching with digital features. However, these applications are not as flexible as paper and often obstruct spontaneous sketching. To overcome these drawbacks we suggest the usage of digital pens and paper in conjunction with sketch-based UML diagram tools. This paper deals with work in progress concerning the seamless integration of paper-based and digital UML-sketching. Amongst others, this includes special UML sketch books and paper palettes used in combination with tabletops.

Editing and exploring node-link diagrams on pen- and multi-touch-operated tabletops

This project addresses the design of interaction techniques for the creation and manipulation of node-link diagrams on multi-touch and pen enabled displays. Analysis and creation of node-link diagrams is an important activity, and one that can benefit greatly from the enhanced interaction bandwidth and collaborative affordances of interactive tabletops. The applications that we will demonstrate implement a broad set of novel interaction techniques for editing and manipulating node-link diagrams. Some techniques have been implemented with hybrid input (pens + touch). They allow flexible creation and manipulation of diagram elements by sketching and structural editing. This includes connecting and copying nodes by bimanual input and changing types of edges by gestures. Other techniques are meant to support users in analyzing diagrams. For example, by strumming or bundling edges, it is easy to see what nodes are connected by the edges.