Giuseppe Raffa

Don't slow me down: Bringing energy efficiency to continuous gesture recognition

Gesture is a compelling user interaction modality for enabling truly on-the-go interactions. Unlike keyboard and touch screen interactions which require considerable visual attention and impose stringent constrains on the form factor of mobile devices, people can easily use hand gestures to perform simple actions (e.g. retrieve voice mail) without having to slow down. In this paper we present an efficient gesture recognition pipeline optimized for “continuous” recognition while minimizing processing overhead and enhancing usability by not requiring the user to delimit explicitly the start and end of gestures. The pipeline is constructed to allow for early filtering of unwanted sensor data with minimal processing cost, and limiting the invocation of processing intensive stages (i.e. HMM) to a limited subset of data (< 5% of sensor data). We also present our evaluation results from a 10 user experiment using 17 gestures and demonstrate that we can achieve considerable processing and power saving without impacting overall recognition accuracy.

Energy-efficient mobile gesture recognition with computation offloading

Gesture recognition is a novel and compelling user input modality which allows users to interact quickly and naturally with their devices with less demand on their visual attention. Continuous gesture recognition places stringent demands on device power consumption, battery life and processing capability. In this work, we show that we can reduce the energy consumed during continuous gesture recognition on a mobile device with the delegation of the pre-processing stages, which filter out non-gesture segments, to a low power node that is separate from the main CPU. The main CPU can thus be kept in stop mode until a potential gesture is detected by the low power node, invoking the main processor to perform the computation-intensive gesture classification to detect which exact gesture has been performed by the user. We present details of the processing performance and power consumed at each step of the processing pipeline, showing the extent of power savings achieved. Experiments were conducted for detailed evaluation of the power consumption of the optimized gesture pipeline.

An Extensible Sensor based Inferencing Framework for Context Aware Applications

Development of modern context-aware applications requires the acquisition and interpretation of data from one or more sensors. This paper presents a framework designed to enable developers of such applications to focus on high level design and functionalities rather than spending time in low level implementation details. The framework enables this behavior and tight real-time control of an inferencing workload by representing it with a directed acyclic graph and by providing horizontal capabilities in order to adapt to a number of different usages. The main features enabled by the framework are reusability across algorithms, standard interfaces among them, improved efficiency through easy to use optimization techniques, and parallel processing of different workloads. We will show how the framework can be used to implement a number of disparate workloads that enable a range of context aware use cases, describe its implementation and finally discuss future work.