Sami Ronkainen

Tap input as an embedded interaction method for mobile devices

In this paper we describe a novel interaction method for interacting with mobile devices without the need to access a keypad or a display. A tap with a hand can be reliably detected e.g. through a pocket by means of an acceleration sensor. By carefully designing the user interface, the tap can be used to activate logically similar functionalities on the device, leading to a simple but useful interaction method. We present results of user tests aimed at studying the usability of various tap input based user interface applications.

Utilising context ontology in mobile device application personalisation

Context Studio, an application personalisation tool for semi-automated context-based adaptation, has been proposed to provide a flexible means of implementing context-aware features. In this paper, Context Studio is further developed for the end users of small-screen mobile devices. Navigating and information presentation are designed for small screens, especially for the Series 60 mobile phone user interface. Context ontology, with an enhanced vocabulary model, is utilized to offer scalable representation and easy navigation of context and action information in the UI. The ontology vocabulary hierarchy is transformed into a folder-file model representation in the graphical user interface. UI elements can be directly updated, according to the extensions and modifications to ontology vocabularies, automatically in an online system. A rule model is utilized to allow systematic management and presentation of context-action rules in the user interface. The chosen ontology-based UI model is evaluated with a usability study.

Environment Analysis as a Basis for Designing Multimodal and Multidevice User Interfaces

In this article, we present a practical approach to analyzing mobile usage environments. We propose a framework for analyzing the restrictions that characteristics of different environments pose on the users capabilities. These restrictions along with current user interfaces form the cost of interaction in a certain environment. Our framework aims to illustrate that cost and what causes it. The framework presents a way to map features of the environment to the effects they cause on the resources of the user and in some cases on the mobile device. This information can be used for guiding the design of adaptive and/or multimodal user interfaces or devices optimized for certain usage environments. An example of using the framework is presented along with some major findings and three examples of applying them in user interface design.

Interaction and end-user programming with a context-aware mobile application

In this paper we present the user interface design and evaluation of a tool for customizing mobile phone applications with context-aware features. The tool enables the user to program a set of context-action rules, defining the behavior of the device when a selected context is recognized and/or some other user-defined conditions are met. The tool user interface design is described starting from an early paper prototype and its evaluation, leading to a functional software implementation in a mobile phone. Finally, the usability evaluation of the functional prototype, and other relevant findings from the user test, are presented.

Time-out in user interface: the case of mobile text input

In many user interfaces with restricted input capabilities, a time-out is used to automatically change the user interface (UI) from one mode to another. In this paper, we study the learning of time-outs and the effect of feedback on it in the case of mobile phone text entry. The effects of three different feedback schemes (auditory/visual/no feedback) on the learning of two different time-out lengths were compared. We measured the response time (RT) from the time-out occurrence to the time of the user’s action. Error rates and the development of the RTs in different schemes were used as measures of learning. We also studied if the users learned to estimate the time-out lengths, or if they just reacted to the available feedback. There were three main findings. Without feedback, RTs had a great variation. Auditory feedback enabled faster RTs than visual feedback. Finally, we found evidence of short-term learning, but not much of a lasting effect. The possible application of adapting time-out length to user RT is discussed.

Camera assisted multimodal user interaction

Since more processing power, new sensing and display technologies are already available in mobile devices, there has been increased interest in building systems to communicate via different modalities such as speech, gesture, expression, and touch. In context identification based user interfaces, these independent modalities are combined to create new ways how the users interact with hand-helds. While these are unlikely to completely replace traditional interfaces, they will considerably enrich and improve the user experience and task performance. We demonstrate a set of novel user interface concepts that rely on built-in multiple sensors of modern mobile devices for recognizing the context and sequences of actions. In particular, we use the camera to detect whether the user is watching the device, for instance, to make the decision to turn on the display backlight. In our approach the motion sensors are first employed for detecting the handling of the device. Then, based on ambient illumination information provided by a light sensor, the cameras are turned on. The frontal camera is used for face detection, while the back camera provides for supplemental contextual information. The subsequent applications triggered by the context can be, for example, image capturing, or bar code reading.

Time-Out in Mobile Text Input: The Effects of Learning and Feedback

In many user interfaces with restricted input/output capabilities, a time-out is used to automatically change the UI from one mode into another. In this paper we studied the learning of time-outs and the effect of feedback on it in mobile phone text entry. The effects of three different feedback schemes (auditory/visual/no feedback) on learning of two different time-out lengths were compared. We measured the response time from the time-out occurrence to the time of user’s reaction. Error rates and the development of the response times in different schemes were used as measures of learning. We also studied if the users learned to estimate the time-out lengths, or if they just reacted to the available feedback. There were three main findings. Without feedback, response times had great variation. Auditory feedback enabled faster response times than visual. Finally, we found evidence of short-term learning, but not as much of a lasting effect.