Peter Boda

Physical Activity Recognition with Mobile Phones: Challenges, Methods, and Applications

In this book chapter, we present a novel system that recognizes and records the physical activity of a person using a mobile phone. The sensor data is collected by built-in accelerometer sensor that measures the motion intensity of the device. The system recognizes five everyday activities in real-time, i.e., stationary, walking, running, bicycling, and in vehicle. We first introduce the sensor’s data format, sensor calibration, signal projection, feature extraction, and selection methods. Then we have a detailed discussion and comparison of different choices of feature sets and classifiers. The design and implementation of one prototype system is presented along with resource and performance benchmark on Nokia N95 platform. Results show high recognition accuracies for distinguishing the five activities. The last part of the chapter introduces one demo application built on top of our system, physical activity diary, and a selection of potential applications in mobile wellness, mobile social sharing and contextual user interface domains.

Capricorn: an intelligent interface for mobile widgets

Widgets are embeddable objects that provide easy and ubiquitous access to dynamic information sources, for example weather, news or TV program information. Widgets are typically rather static - they provide the information regardless of whether the information is relevant to the users current information needs. In this paper we introduce Capricorn, which is an intelligent interface for mobile widgets. The interface uses various adaptive web techniques for facilitating navigation. For example, we use collaborative filtering to recommend suitable widgets and we dim infrequently used widgets. The demonstration presents the Capricorn interface focusing on the adaptive parts of the interface. The user interface is web-based, and as such platform independent. However, our target environment is mobile phones, and thus the interface has been optimized for mobile phones.

Capricorn - an intelligent user interface for mobile widgets

Widgets are embeddable objects that provide easy and ubiquitous access to dynamic information sources, e.g., weather, news or TV program information. Interactions with widgets take place through a so-called widget engine, which is a specialized client-side runtime component that also provides functionalities for managing widgets. As the number of supported widgets increases, managing widgets becomes increasingly complex. For example, finding relevant or interesting widgets becomes difficult and the user interface easily gets cluttered with irrelevant widgets. In addition, interacting with information sources can be cumbersome, especially on mobile platforms. In order to facilitate widget management and interactions, we have developed Capricorn, an intelligent user interface that integrates adaptive navigation techniques into a widget engine. This paper describes the main functionalities of Capricorn and presents the results of a usability evaluation that measured user satisfaction and compared how user satisfaction varies between desktop and mobile platforms.

A contextual multimodal integrator

Multimodal Integration addresses the problem of combining various user inputs into a single semantic representation that can be used in deciding the next step of system action(s). The method presented in this paper uses a statistical framework to implement the integration mechanism and includes contextual information additionally to the actual user input. The underlying assumption is that the more information sources are taken into account, the better picture can be drawn about the actual intention of the user in the given context of the interaction. The paper presents the latest results with a Maximum Entropy classifier, with special emphasis on the use of contextual information (type of gesture movements and type of objects selected). Instead of explaining the design and implementation process in details (a longer paper to be published later will do that), only a short description is provided here about the demonstration implementation that produces above 91% accuracy for the 1st best and higher than 96% for the accumulated five N-bests results.

A maximum entropy based approach for multimodal integration

Integration of various user input channels for a multimodal interface is not just an engineering problem. To fully understand users in the context of an application and the current session, solutions are sought that process information from different intentional, i.e. user-originated, as well as from passively available sources in a uniform manner. As a first step towards this goal, the work demonstrated here investigates how intentional user input (e.g. speech, gesture) can be seamlessly combined to provide a single semantic interpretation of the user input. For this classical Multimodal Integration problem the Maximum Entropy approach is demonstrated with 76.52% integration accuracy for the 1st and 86.77% accuracy for the top 3-best candidates. The paper also exhibits the process that generates multimodal data for training the statistical integrator, using transcribed speech from MITs Voyager application. The quality of the generated data is assessed by comparing to real inputs to the multimodal version of Voyager.

PassItOn: An Opportunistic Messaging Prototype on Mobile Devices

With the increasing popularity of mobile handheld devices and the growing capability of these devices, it is becoming possible that information sharing/dissemination is carried out through human networks, as a complement to the traditional computer networks. In such human networks, people come across one another, while their mobile devices exchange and store information in a spontaneous and transparent way. Such an encounter could be established through direct device-todevice connectivity when two devices come into each other’s communication range, or be enabled by, e.g., a Wi-Fi access point, when the devices both enter its coverage. A new form of dissemination, which we call opportunistic messaging, is such an application that is based on human encounters and mobilities. When human encounters are exploited for communications, the reliance on network infrastructure access is eliminated; communications can be performed even where infrastructure is absent or infrastructure access is intermittent. By leveraging human mobilities, data delivery does not require an end-to-end path from the source to a recipient; instead, people carrying mobile devices serve as relays – they cache others’ data and forward/deliver the data when appropriate. Thus, the propagation of information is tied to people’s physical proximity when they move around, and incorporates the social aspects of communications as people tend to spend more time co-locating with their social relations. Opportunistic messaging is applicable anywhere, and is especially appealing where network infrastructure access is limited or intermittent (e.g., on cruise ships, in national parks, after disasters). Another intriguing characteristic of it is its ease of deployment – no central server is needed, but only a single piece of software on users’ mobile devices. However, as human encounters and mobilities are unpredictable, when used for social applications, opportunistic messaging is most suited for disseminating user-generated information that is non-formal, less important, and thus not time-critical. In recent years, a considerable amount of efforts have been invested in the research on opportunistic networking and delay-tolerant networking (which encompasses opportunistic networking but is a broader concept). A large portion of prior work has focused on routing issues, e.g., through whom as intermediate carriers to deliver a message to the destinations [1] [2] [3] [4]. The routing issues have been further explored in various contexts, such as in vehicular networks [5] [6] [7] and in social networking applications [8] [9] [10] [11]. However, serious real-world application development, deployment and evaluation of the opportunistic networking concepts still fall behind [12], in which many challenging issues remain to be addressed (to name a few, location-awareness, user incentives and preferences, power preservation, encounter controls, etc.). In this work, we design and prototype PassItOn, a fully distributed opportunistic messaging system. Our goal is to build up a proof-of-concept platform on real mobile devices, and thus show the feasibility and potentials of utilizing human movements for dissemination applications. Meanwhile, we seek to shed lights on the design, implementation and deployment issues in building such systems, and thus stimulate new ideas and perspectives on addressing these issues. Moreover, we aim to offer a real testbed on which new mechanisms, protocols and use cases can be tested and evaluated.

WebCall - A Rich Context Mobile Research Framework

The ever-increasing capability of mobile devices enables many mobile services far beyond a traditional voice call. In this paper, we present WebCall, a research framework on how to share and utilize the rich contextual information about users, such as phonebook, indoor and outdoor location, and calendar. WebCall also demonstrates a few services (e.g. human powered questions and answers) that can be built on top of user context. Third-party services can be integrated with WebCall through a simple API and potentially benefit from context filtering. An invitation mechanism is introduced to enable bootstrapping user base. Privacy concern is addressed by giving full control to users on how to share their information.

Ad-hoc Networks Extending Wireless Broadband Coverage and Capacity for Mobile Internet Usage

The number of wireless Internet users is expected to increase rapidly within the next few years resulting in a huge increase in the number of bits transferred between wireless devices and networks. When looking at current mobile networks, they have mainly been designed for optimized delivery of voice calls and not for broadband data. Although mobile networks are evolving, we expect there to be a need for complementary solutions for providing wireless Internet access. Thus we introduce an ad-hoc networking solution using WiFi short range radio technology for extending cellular wireless broadband coverage and capacity into places where it is most urgently needed; densely populated areas and indoors. We further propose connecting the ad-hoc networking solution to the operators total access offering. As a result, wireless users may enjoy an easy to use, good quality, secure and robust service offering when entering the wireless Internet era, and access providers are equipped with tools to utilize complementary access technologies in places where they suit best. Operators arc able to make efficient use of their existing investments and open new service concepts and business models with mobile Internet domain use cases evolution. Also end users become an essential part of business ecosystem by providing content, services and access to other users.