Enrico Rukzio

An experimental comparison of physical mobile interaction techniques: touching, pointing and scanning

This paper presents an analysis, implementation and evaluation of the physical mobile interaction techniques touching, pointing and scanning. Based on this we have formulated guidelines that show in which context which interaction technique is preferred by the user. Our main goal was to identify typical situations and scenarios in which the different techniques might be useful or not. In support of these aims we have developed and evaluated, within a user study, a low-fidelity and a high-fidelity prototype to assess scanning, pointing and touching interaction techniques within different contexts. Other work has shown that mobile devices can act as universal remote controls for interaction with smart objects but, to date, there has been no research which has analyzed when a given mobile interaction technique should be used. In this research we analyze the appropriateness of three interaction techniques as selection techniques in smart environments.

Touch & interact: touch-based interaction of mobile phones with displays

The limited screen size and resolution of current mobile devices can still be problematic for map, multimedia and browsing applications. In this paper we present Touch & Interact: an interaction technique in which a mobile phone is able to touch a display, at any position, to perform selections. Through the combination of the output capabilities of the mobile phone and display, applications can share the entire display space. Moreover, there is potential to realize new interaction techniques between the phone and display. For example, select & pick and select & drop are interactions whereby entities can be picked up onto the phone or dropped onto the display. We report the implementation of Touch & Interact, its usage for a tourist guide application and experimental comparison. The latter shows that the performance of Touch & Interact is comparable to approaches based on a touch screen; it also shows the advantages of our system regarding ease of use, intuitiveness and enjoyment.

PhoneTouch: a technique for direct phone interaction on surfaces

PhoneTouch is a novel technique for integration of mobile phones and interactive surfaces. The technique enables use of phones to select targets on the surface by direct touch, facilitating for instance pick&drop-style transfer of objects between phone and surface. The technique is based on separate detection of phone touch events by the surface, which determines location of the touch, and by the phone, which contributes device identity. The device-level observations are merged based on correlation in time. We describe a proof-of-concept implementation of the technique, using vision for touch detection on the surface (including discrimination of finger versus phone touch) and acceleration features for detection by the phone.

Investigating selection and reading performance on a mobile phone while walking

More and more people interact with their mobile phone while walking. The presented research analyzes; firstly, the negative effect of walking when considering reading and target selection tasks, such as weaker performance and higher workload. Here, we focused on one-handed interaction with a touch screen whereby the thumb is used as the input device. Secondly, we analyze how these negative effects can be compensated by increasing the text size and the size of the targets to select on the mobile phone. A comparative user study was conducted with 16 participants who performed target acquisition and reading tasks while standing and walking. The results show that whilst performance decreases, cognitive load increases significantly when reading and selecting targets when walking. Furthermore, the results show that the negative effect regarding target selection can be compensated by increasing the target size, but the text reading task did not yield better performance results for a larger text size due to the increased demand for scrolling. These results can be used to inform future designs of mobile user interfaces which might provide a dedicated walking mode.

A cross-device interaction style for mobiles and surfaces

Natural forms of interaction have evolved for personal devices that we carry with us (mobiles) as well as for shared interactive displays around us (surfaces) but interaction across the two remains cumbersome in practice. We propose a novel cross-device interaction style for mobiles and surfaces that uses the mobile for tangible input on the surface in a stylus-like fashion. Building on the direct manipulation that we can perform on either device, it facilitates fluid and seamless interaction spanning across device boundaries. We provide a characterization of the combined interaction style in terms of input, output, and contextual attributes, and demonstrate its versatility by implementation of a range of novel interaction techniques for mobile devices on interactive surfaces.

Mobile interaction with the real world: an evaluation and comparison of physical mobile interaction techniques

Mobile devices are more and more used for mobile interactions with things, places and people in the real world. However, so far no studies have discussed which interaction techniques are preferred by users in different contexts. This paper presents an experimental comparison of four different physical mobile interaction techniques: touching, pointing, scanning and user-mediated object interaction. To evaluate these techniques across different scenarios and to collect real usage data, four prototypes were implemented: a system for mobile interaction in smart environments, a mobile tourist guide, a mobile museum guide and a prototype for mobile interaction with advertisement posters. In each setting an experimental comparison was performed. Based on the results of these studies, which involved over 60 participants in total, advantages and disadvantages of these interaction techniques are described. Context-specific user preferences are presented for the interaction techniques, to help application designers and developers decide which interaction technique(s) to integrate into their application and which consequences this decision has.

Supporting Mobile Service Usage through Physical Mobile Interaction

Although mobile services can be used ubiquitously, their employment and the interaction with them are still restricted by the constraints of mobile devices. In order to facilitate and leverage mobile interaction with services, we present a generic framework that combines semantic Web service technology and physical mobile interaction. This interaction paradigm uses mobile devices to extract information from augmented physical objects and use it for a more intuitive and convenient invocation of associated services. For that purpose, the presented framework exploits Web service descriptions for the automatic and dynamic generation of customizable user interfaces that support and facilitate physical mobile interaction. This generic approach to mobile interaction with services through the interaction with physical objects promises to meet the complementary development of the Internet of things. A user study with a prototype application for mobile ticketing confirms our concept and shows its limits

Design, implementation and evaluation of a novel public display for pedestrian navigation: the rotating compass

Important drawbacks of map-based navigation applications for mobile phones are their small screen size and that users have to associate the information provided by the mobile phone with the real word. Therefore, we designed, implemented and evaluated the Rotating Compass - a novel public display for pedestrian navigation. Here, a floor display continuously shows different directions (in a clockwise order) and the mobile phone informs the user when their desired direction is indicated. To inform the user, the mobile phone vibrates in synchronization with the indicated direction. We report an outdoor study that compares a conventional paper map, a navigation application running on a mobile device, navigation information provided by a public display, and the Rotating Compass. The results provide clear evidence of the advantages of the new interaction technique when considering task completion time, context switches, disorientation events, usability satisfaction, workload and multi-user support.

Supporting device discovery and spontaneous interaction with spatial references

The RELATE interaction model is designed to support spontaneous interaction of mobile users with devices and services in their environment. The model is based on spatial references that capture the spatial relationship of a user’s device with other co-located devices. Spatial references are obtained by relative position sensing and integrated in the mobile user interface to spatially visualize the arrangement of discovered devices, and to provide direct access for interaction across devices. In this paper we discuss two prototype systems demonstrating the utility of the model in collaborative and mobile settings, and present a study on usability of spatial list and map representations for device selection.

Projector phone: a study of using mobile phones with integrated projector for interaction with maps

First working prototypes of mobile phones with integrated pico projectors have already been demonstrated and it is expected that such projector phones will be sold within the next three years. Applications that require interaction with large amounts of information will benefit from the large projection and its high resolution. This paper analyses the advantages and disadvantages of an integrated projector when interacting with maps, and discusses findings useful for the development of mobile applications for projector phones. We report in particular the implementation of an application that uses either the screen of the mobile phone, the projection or a combination of both. These three options were compared in a user study in which the participants had to perform three different tasks with each option. The results provide clear evidence for the positive aspects of using a built-in projector, but also show some negative aspects related to text input.