Sanna Kallio

Accelerometer-based gesture control for a design environment

Accelerometer-based gesture control is studied as a supplementary or an alternative interaction modality. Gesture commands freely trainable by the user can be used for controlling external devices with handheld wireless sensor unit. Two user studies are presented. The first study concerns finding gestures for controlling a design environment (Smart Design Studio), TV, VCR, and lighting. The results indicate that different people usually prefer different gestures for the same task, and hence it should be possible to personalise them. The second user study concerns evaluating the usefulness of the gesture modality compared to other interaction modalities for controlling a design environment. The other modalities were speech, RFID-based physical tangible objects, laser-tracked pen, and PDA stylus. The results suggest that gestures are a natural modality for certain tasks, and can augment other modalities. Gesture commands were found to be natural, especially for commands with spatial association in design environment control.

Online gesture recognition system for mobile interaction

This paper introduces an accelerometer-based online gesture recognition system. Recognition of gestures can be utilised as a part of a human computer interaction for mobile devices, e.g. cell phones, PDAs and remote controllers. Gestures are captured with a small wireless sensor-box that produces three dimensional acceleration signal. Acceleration signal is preprocessed, vector quantised and finally classified using Hidden Markov Models. The design of online gesture recognition for mobile devices sets requirements for data processing. Thus, the system uses a small size codebook and simple preprocessing methods. The recognition accuracy of system is tested with gestures of four degrees of complexity. Experimental results show great potential for recognising simple and even more complex gestures with good accuracy.

USER INDEPENDENT GESTURE INTERACTION FOR SMALL HANDHELD DEVICES

Accelerometer-based gesture recognition facilitates a complementary interaction modality for controlling mobile devices and home appliances. Using gestures for the task of home appliance control requires use of the same device and gestures by different persons, i.e. user independent gesture recognition. The practical application in small embedded low-resource devices also requires high computational performance. The user independent gesture recognition accuracy was evaluated with a set of eight gestures and seven users, with a total of 1120 gestures in the dataset. Twenty-state continuous HMM yielded an average of 96.9% user independent recognition accuracy, which was cross-validated by leaving one user in turn out of the training set. Continuous and discrete five-state HMM computational performances were compared with a reference test in a PC environment, indicating that discrete HMM is 20% faster. Computational performance of discrete five-state HMM was evaluated in an embedded hardware environment with a 104 MHz ARM-9 processor and Symbian OS. The average recognition time per gesture calculated from 1120 gesture repetitions was 8.3 ms. With this result, the computational performance difference between the compared methods is considered insignificant in terms of practical application. Continuous HMM is hence recommended as a preferred method due to its better suitability for a continuous-valued signal, and better recognition accuracy. The results suggest that, according to both evaluation criteria, HMM is feasible for practical user independent gesture control applications in mobile low-resource embedded environments.

Visualization of hand gestures for pervasive computing environments

Visualization method is proposed as an additional feature for accelerometer-based gesture control. The motivation for visualization of gesture control is justified and the challenges related to visualization are presented. The gesture control is based on Hidden Markov Models. This paper describes basic concepts of the gesture visualization and studies how well the developed visualization method can animate hand movement performed during the gesture control. The results indicate that visualization clearly provides information about the performed gesture, and it could be utilized in providing essential feedback and guidance to the user in future gesture control applications.

A layered approach to context-dependent user modelling

This works presents a method for explicit acquisition of context-dependent user preferences (preferences which change depending on a user situation, e.g., higher interest in outdoor activities if it is sunny than if it is raining) for Smart Home - intelligent environment, which recognises contexts of its inhabitants (such as presence of people, activities, events, weather etc) via home and mobile devices and provides personalized proactive support to the users. Since a set of personally important situations, which affect user preferences, is user-dependent, and since many situations can be described only in fuzzy terms, we provide users with an easy way to develop personal context ontology and to map it fuzzily into common ontology via GUI. Backward mapping, by estimating the probability of occurrence of a user-defined situation, allows retrieval of preferences from all components of the user model.