Julian Seifert

A cross-device interaction style for mobiles and surfaces

Natural forms of interaction have evolved for personal devices that we carry with us (mobiles) as well as for shared interactive displays around us (surfaces) but interaction across the two remains cumbersome in practice. We propose a novel cross-device interaction style for mobiles and surfaces that uses the mobile for tangible input on the surface in a stylus-like fashion. Building on the direct manipulation that we can perform on either device, it facilitates fluid and seamless interaction spanning across device boundaries. We provide a characterization of the combined interaction style in terms of input, output, and contextual attributes, and demonstrate its versatility by implementation of a range of novel interaction techniques for mobile devices on interactive surfaces.

MobiSurf: improving co-located collaboration through integrating mobile devices and interactive surfaces

One of the most popular scenarios for advertising interactive surfaces in the home is their support for solving co-located collaborative tasks. Examples include joint planning of events (e.g., holidays) or deciding on a shared purchase (e.g., a present for a common friend). However, this usually implies that all interactions with information happen on the common display. This is in contrast to the current practices to use personal devices and further, most peoples behavior to constantly switch between individual and group phases because people have differing search strategies, preferences, etc. We therefore investigated how the combination of personal devices and a simple way of exchanging information between these devices and an interactive surface changes the way people solve collaborative tasks compared to an existing approach of using personal devices. Our study results clearly indicate that the combination of personal and a shared device allows users to fluently switch between individual and group work phases and users take advantage of both device classes.

Pervasive information through constant personal projection: the ambient mobile pervasive display (AMP-D)

The vision of pervasive ambient information displays which show relevant information has not yet come true. One of the main reasons is the limited number of available displays in the environment which is a fundamental requirement of the original vision. We introduce the concept of an Ambient Mobile Pervasive Display AMP-D which is a wearable projector system that constantly projects an ambient information display in front of the user. The floor display provides serendipitous access to public and personal information. The display is combined with a projected display on the users hand, forming a continuous interaction space that is controlled by hand gestures. The paper introduces this novel device concept, discusses its interaction design, and explores its advantages through various implemented application examples. Furthermore, we present the AMP-D prototype which illustrates the involved challenges concerning hardware, sensing, and visualization.

Mobidev: a tool for creating apps on mobile phones

Currently, the development of mobile applications heavily relies on using conventional computers as development platform. MobiDev enables people in emerging countries without access to a computer but to a cell phone to develop their own locally relevant applications. The goal of the Mo-biDev project is to simplify development and deployment of applications directly on mobile phones. As a first step, we focus on the design of applications and try to support the computer science curriculum in developing countries to bootstrap the mobile developer culture and community. MobiDev allows the creation of graphical user interfaces (GUI) using various concepts. We present the results of a first system evaluation that show how people perceive the concepts for UI creation of MobiDev.

PointerPhone: Using Mobile Phones for Direct Pointing Interactions with Remote Displays

Large screens or projections in public and private settings have become part of our daily lives, as they enable the collaboration and presentation of information in many diverse ways. When discussing the shown information with other persons, we often point to a displayed object with our index finger or a laser pointer in order to talk about it. Although mobile phone-based interactions with remote screens have been investigated intensively in the last decade, none of them considered such direct pointing interactions for application in everyday tasks. In this paper, we present the concept and design space of PointerPhone which enables users to directly point at objects on a remote screen with their mobile phone and interact with them in a natural and seamless way. We detail the design space and distinguish three categories of interactions including low-level interactions using the mobile phone as a precise and fast pointing device, as well as an input and output device. We detail the category of widget-level interactions. Further, we demonstrate versatile high-level interaction techniques and show their application in a collaborative presentation scenario. Based on the results of a qualitative study, we provide design implications for application designs.

Hover Pad: interacting with autonomous and self-actuated displays in space

Handheld displays enable flexible spatial exploration of information spaces -- users can physically navigate through three-dimensional space to access information at specific locations. Having users constantly hold the display, however, has several limitations: (1) inaccuracies due to natural hand tremors; (2) fatigue over time; and (3) limited exploration within arms reach. We investigate autonomous, self-actuated displays that can freely move and hold their position and orientation in space without users having to hold them at all times. We illustrate various stages of such a displays autonomy ranging from manual to fully autonomous, which -- depending on the tasks -- facilitate the interaction. Further, we discuss possible motion control mechanisms for these displays and present several interaction techniques enabled by such displays. Our Hover Pad toolkit enables exploring five degrees of freedom of self-actuated and autonomous displays and the developed control and interaction techniques. We illustrate the utility of our toolkit with five prototype applications, such as a volumetric medical data explorer.

UbiBeam: An Interactive Projector-Camera System for Domestic Deployment

Previous research on projector-camera systems has focused for a long time on interaction inside a lab environment. Currently they are no insight on how people would interact and use such a device in their everyday lives. We conducted an in-situ user study by visiting 22 households and exploring specific use cases and ideas of portable projector-camera systems in a domestic environment. Using a grounded theory approach, we identified several categories such as interaction techniques, presentation space, placement and use cases. Based on our observations, we designed and implement UbiBeam, a domestically deployable projector-camera system. The system comprises a projector, a depth camera and two servomotors to transform every ordinary surface into a touch-sensitive information display.

Penbook: bringing pen+paper interaction to a tablet device to facilitate paper-based workflows in the hospital domain

In many contexts, pen and paper are the ideal option for collecting information despite the pervasiveness of mobile devices. Reasons include the unconstrained nature of sketching or handwriting, as well as the tactility of moving a pen over a paper that supports very fine granular control of the pen. In particular in the context of hospitals, many writing and note taking tasks are still performed using pen and paper. However, often this requires time-consuming transcription into digital form for the sake of documentation. We present Penbook - a system providing a touch screen together with a built-in projector integrated with a wireless pen and a projection screen augmented with Anoto paper. This allows using the pen to write or sketch digital information with light on the projection surface while having the distinct tactility of a pen moving over paper. The touch screen can be used in parallel with the projected information turning the tablet into a dual-display device. In this paper, we present the Penbook concept, detail specific applications in a hospital context, and present a prototype implementation of Penbook.

PermissionWatcher: Creating User Awareness of Application Permissions in Mobile Systems

Permission systems control access of mobile applications to other applications, data, and resources on a smartphone. Both from a technical and a social point of view, they are based on the assumption that users actually understand these permissions and hence they can make an informed decision about which permission to grant to which piece of software. Results of a survey conducted for this article seriously challenges this assumption. For instance, over a third of participating Android users were not able to correctly identify the meaning of the permission Full Internet Access. We developed PermissionWatcher, an Android application which provides users with awareness information about other applications and allows to check on the permission set granted to individual applications. In a field study with 1000+ Android users, we collected data that provides evidence that users are willing to follow security principles if security awareness is created and information is presented in a clear and comprehensive way. Therefore, we argue that it is essential for security policies to take the abilities of the target audience into account.

From the private into the public: privacy-respecting mobile interaction techniques for sharing data on surfaces

Interactive horizontal surfaces provide large semi-public or public displays for colocated collaboration. In many cases, users want to show, discuss, and copy personal information or media, which are typically stored on their mobile phones, on such a surface. This paper presents three novel direct interaction techniques (Select&Place2Share, Select&Touch2Share, and Shield&Share) that allow users to select in private which information they want to share on the surface. All techniques are based on physical contact between mobile phone and surface. Users touch the surface with their phone or place it on the surface to determine the location for information or media to be shared. We compared these three techniques with the most frequently reported approach that immediately shows all media files on the table after placing the phone on a shared surface. The results of our user study show that such privacy-preserving techniques are considered as crucial in this context and highlight in particular the advantages of Select&Place2Share and Select&Touch2Share in terms of user preferences, task load, and task completion time.