Janne Bergman

Gravity sphere: gestural audio-tactile interface for mobile music exploration

Current solutions for managing music in mobile contexts are inconvenient as they require considerable effort and visual attention. We describe a novel system for exploring music and generating playlists in mobile contexts, and present findings from our formative usability evaluations. The system provides audio-tactile feedback and is controlled by manipulating a devices orientation. The system plays songs associated with the current orientation. A spatial gesture or other command is then used to lock the orientation into a playlist. We evaluated two iterations of a prototype of the system and found that users were successful in exploring music and generating playlists with the system. We found that certain orientations are more common than others. Also, manipulating the prototype felt more natural while walking than sitting. Personalization of the music mapping was requested by users and seen as beneficial for usability.

CATCH-2004 multi-modal browser: overview description with usability analysis

This paper takes a closer look at the user interface issues in our research multi-modal browser architecture. The browser framework, also briefly introduced in this paper, reuses single-modal browser technologies available for VoiceXML, WML, and HTML browsing. User interface actions on a particular browser are captured, converted to events, and distributed to the other browsers participating (possibly on different hosts) in the multi-modal framework. We have defined a synchronization protocol, which distributes such events with the help of the central component called the Virtual Proxy. The choice of the architecture and the synchronization primitives have profound consequences on handling certain interesting UI use cases. We particularly address those specified by the W3C MultiModal Requirements, which are related to the design of possible strategies of dealing with simultaneous input, solving input inconsistencies, and defining synchronization points. The proposed approaches are illustrated by examples.

Loosely-coupled approach towards multi-modal browsing

Contemplating the concept of universal-access multi-modal browsing comes as one of the emerging “killer” technologies that promises broader and more flexible access to information, faster task completion, and advanced user experience. Inheriting the best from GUI and speech, based on the circumstances, hardware capabilities, and environment, multi-modality’s great advantage is to provide application developers with a scalable blend of input and output channels that may accommodate any user, device, and platform. This article describes a flexible multi-modal browser architecture, named Ferda the Ant, which reuses uni-modal browser technologies available for VoiceXML, WML, and HTML browsing. A central component, the Virtual Proxy, acts as a synchronization coordinator. This browser architecture can be implemented in either a single client configuration, or by distributing the browser components across the network. We have defined and implemented a synchronization protocol to communicate the changes occurring in the context of a component browser to the other browsers participating in the multi-modal browser framework. Browser wrappers implement the required synchronization protocol functionality at each of the component browsers. The component browsers comply with existing content authoring standards, and we have designed a set of markup-level authoring conventions that facilitate maintaining the browser synchronization .

Interacting with the flow

Mobile devices offer challenges for UI design. Limited screen space leads to deep menus, complex navigation and loss of position. We introduce a new user interface concept that reverses the traditional navigation paradigm. By utilizing context awareness and allowing the user to control the UI via filters, objects of interest navigate past the user instead of the user navigating to the object. The user operates on a single view without the need for deep menu navigation. The new UI is also easy to configure. We implemented the concept on the Nokia S60 5th edition touch platform and conducted user testing with 16 users. Initially, users felt confused because of new ways of accessing things. However, after a short period of usage, majority of the users found it easy to use. Most of the users felt the system to be fun and playful.