Beat Signer

PaperPoint: a paper-based presentation and interactive paper prototyping tool

Recent developments in digital pen and paper solutions enable, not only the digital capture of handwriting, but also paper to be used as an interactive medium that links to digital information and services. We present a tool that builds on technologies for interactive paper to enable PowerPoint presentations to be controlled from printed slide handouts. Furthermore, slides can be easily annotated during presentations by simply drawing on the printed version of the slide. As well as discussing the advantages of such a paper-based interface and initial findings on its use, we describe how we were also able to exploit it to provide a general prototyping tool for interactive paper applications.

Only touching the surface: creating affinities between digital content and paper

Despite the wide-ranging recognition that paper remains a pervasive resource for human conduct and collaboration, there has been uncertain progress in developing technologies to bridge the paper-digital divide. In this essay we discuss the design of a technology that interweaves developments in new materials, electronics and software, and seeks to provide a cheap and accessible solution to creating new affinities between digital content, in whatever form, and ordinary paper. The technology and its design draws from a broad range of field studies, including research in classrooms and museums. These delineate the requirements and considerations that inform solutions to enhancing paper whilst preserving its integrity. The paper also discusses a naturalistic experiment, an evaluation in a museum, where we assessed the technology and the solution. We also chart the progressive development of this solution and the ways in which seemingly simple actions and issues became reconstituted as highly complex technical and analytic problems.

iGesture: A General Gesture Recognition Framework

With the emergence of digital pen and paper interfaces, there is a need for gesture recognition tools for digital pen input. While there exists a variety of gesture recognition frameworks, none of them addresses the issues of supporting application developers as well as the designers of new recognition algorithms and, at the same time, can be integrated with new forms of input devices such as digital pens. We introduce iGesture, a Java-based gesture recognition framework focusing on extensibility and cross-application reusability by providing an integrated solution that includes tools for gesture recognition as well as the creation and management of gesture sets for the evaluation and optimisation of new or existing gesture recognition algorithms. In addition to traditional screen-based interaction, iGesture provides a digital pen and paper interface.

Midas: a declarative multi-touch interaction framework

Over the past few years, multi-touch user interfaces emerged from research prototypes into mass market products. This evolution has been mainly driven by innovative devices such as Apples iPhone or Microsofts Surface tabletop computer. Unfortunately, there seems to be a lack of software engineering abstractions in existing multi-touch development frameworks. Many multi-touch applications are based on hard-coded procedural low level event processing. This leads to proprietary solutions with a lack of gesture extensibility and cross-application reusability. We present Midas, a declarative model for the definition and detection of multi-touch gestures where gestures are expressed via logical rules over a set of input facts. We highlight how our rule-based language approach leads to improvements in gesture extensibility and reusability. Last but not least, we introduce JMidas, an instantiation of Midas for the Java programming language and describe how JMidas has been applied to implement a number of innovative multi-touch gestures.

Paperproof: a paper-digital proof-editing system

We present PaperProof, a paper-digital proof-editing application that allows users to edit digital documents by means of gesture-based mark-up of their printed versions. This enables users to switch seamlessly back and forth between paper and digital instances of a document throughout the document lifecycle, working with whichever medium is preferred for a given task. Importantly, by maintaining a logical mapping between the printed and digital instances, editing operations on paper can later be integrated into the digital document even if other users have edited the digital version in parallel. The system is based on Anoto digital pen and paper technology and is implemented using the iPaper framework for interactive paper.

Mudra: a unified multimodal interaction framework

In recent years, multimodal interfaces have gained momentum as an alternative to traditional WIMP interaction styles. Existing multimodal fusion engines and frameworks range from low-level data stream-oriented approaches to high-level semantic inference-based solutions. However, there is a lack of multimodal interaction engines offering native fusion support across different levels of abstractions to fully exploit the power of multimodal interactions. We present Mudra, a unified multimodal interaction framework supporting the integrated processing of low-level data streams as well as high-level semantic inferences. Our solution is based on a central fact base in combination with a declarative rule-based language to derive new facts at different abstraction levels. Our innovative architecture for multimodal interaction encourages the use of software engineering principles such as modularisation and composition to support a growing set of input modalities as well as to enable the integration of existing or novel multimodal fusion engines.

Print-n-link: weaving the paper web

Citations form the basis for a web of scientific publications. Search engines, embedded hyperlinks and digital libraries all simplify the task of finding publications of interest on the web and navigating to cited publications or web sites. However the actual reading of publications often takes place on paper and frequently on the move. We present a system Print-n-Link that uses technologies for interactive paper to enhance the reading process by enabling users to access digital information and/or searches for cited documents from a printed version of a publication using a digital pen for interaction. A special virtual printer driver automatically generates links from paper to digital services during the printing process based on an analysis of PDF documents. Depending on the user setting and interaction gesture, the system may retrieve metadata about the citation and inform the user through an audio channel or directly display the cited document on the users screen.

Digital annotation of printed documents

We present a general model and information server for the digital annotation of printed documents. The resulting annotation framework supports both informal and structured annotations as well as context-dependent services. A demonstrator application for mammography that features both enhanced writing and reading activities is described.

DelosDLMS - the integrated DELOS digital library management system

DelosDLMS is a prototype of a next-generation Digital Library (DL) management system. It is realized by combining various specialized DL functionalities provided by partners of the DELOS network of excellence. Currently, DelosDLMS combines text and audio-visual searching, offers new information visualization and relevance feedback tools, provides novel interfaces, allows retrieved information to be annotated and processed, integrates and processes sensor data streams, and finally, from a systems engineering point of view, is easily configured and adapted while being reliable and scalable. The prototype is based on the OSIRIS/ISIS platform, a middleware environment developed by ETH Zurich and now being extended at the University of Basel.

As we may link: a general metamodel for hypermedia systems

Many hypermedia models have been proposed, including those specifically developed to model navigational aspects of web sites. But few hypermedia systems have been implemented based on metamodelling principles familiar to the database community. Often there is no clear separation between conceptual and technical issues in the models and their implementations are not based on an explicit representation of a metamodel. This results in a loss of generality and uniformity across systems. Based on principles of metamodel-driven system development, we have implemented a platform that can support various categories of hypermedia systems through the generality and extensibility of the meta-model. We present our metamodel and show how it generalises concepts present in a range of hypermedia and link server systems.