Ulf Blanke

A tutorial on human activity recognition using body-worn inertial sensors

The last 20 years have seen ever-increasing research activity in the field of human activity recognition. With activity recognition having considerably matured, so has the number of challenges in designing, implementing, and evaluating activity recognition systems. This tutorial aims to provide a comprehensive hands-on introduction for newcomers to the field of human activity recognition. It specifically focuses on activity recognition using on-body inertial sensors. We first discuss the key research challenges that human activity recognition shares with general pattern recognition and identify those challenges that are specific to human activity recognition. We then describe the concept of an Activity Recognition Chain (ARC) as a general-purpose framework for designing and evaluating activity recognition systems. We detail each component of the framework, provide references to related research, and introduce the best practice methods developed by the activity recognition research community. We conclude with the educational example problem of recognizing different hand gestures from inertial sensors attached to the upper and lower arm. We illustrate how each component of this framework can be implemented for this specific activity recognition problem and demonstrate how different implementations compare and how they impact overall recognition performance.

Saving Human Lives: What Complexity Science and Information Systems can Contribute

We discuss models and data of crowd disasters, crime, terrorism, war and disease spreading to show that conventional recipes, such as deterrence strategies, are often not effective and sufficient to contain them. Many common approaches do not provide a good picture of the actual system behavior, because they neglect feedback loops, instabilities and cascade effects. The complex and often counter-intuitive behavior of social systems and their macro-level collective dynamics can be better understood by means of complexity science. We highlight that a suitable system design and management can help to stop undesirable cascade effects and to enable favorable kinds of self-organization in the system. In such a way, complexity science can help to save human lives.

Capturing crowd dynamics at large scale events using participatory GPS-localization

Large-scale festivals with a multitude of stages, food stands, and attractions require a complex perimeter design and program planning in order to manage the mobility of crowds as a controlled process. Errors in the planning phase can cause unexpected crowd dynamics and lead to stampedes with lethal consequences. We deployed an official app for Züri Fäscht 2013 - the largest Swiss event - over a period of three days. The app offered information about the festival and featured a background localization allowing us to collect continuously the visitor position. With 56,000 app downloads and 28,000 users contributing 25M location updates in total, we obtained a large scale dataset. By aggregation of location points complex crowd dynamics can be captured during the entire festival. In this paper we present the data collection for Züri Fäscht 2013 and best practices to acquire as many contributing users as possible for such an event. Furthermore, we show the potential of aggregated location data and visualize relevant parameters that can serve as tool for analysis and planning of program and perimeter design.

The telepathic phone: Frictionless activity recognition from WiFi-RSSI

We investigate the use of WiFi Received Signal Strength Information (RSSI) at a mobile phone for the recognition of situations, activities and gestures. In particular, we propose a device-free and passive activity recognition system that does not require any device carried by the user and uses ambient signals. We discuss challenges and lessons learned for the design of such a system on a mobile phone and propose appropriate features to extract activity characteristics from RSSI. We demonstrate the feasibility of recognising activities, gestures and environmental situations from RSSI obtained by a mobile phone. The case studies were conducted over a period of about two months in which about 12 hours of continuous RSSI data was sampled, in two countries and with 11 participants in total. Results demonstrate the potential to utilise RSSI for the extension of the environmental perception of a mobile device as well as for the interaction with touch-free gestures. The system achieves an accuracy of 0.51 while distinguishing as many as 11 gestures and can reach 0.72 on average for four more disparate ones.

GaitAssist: a daily-life support and training system for parkinson's disease patients with freezing of gait

Patients with Parkinsons disease often experience freezing of gait, which bears a high risk of falling, a prevalent cause for morbidity and mortality. In this work we present GaitAssist, a wearable system for freezing of gait support in daily life. The system provides real-time auditory cueing after the onset of freezing episodes. Furthermore, GaitAssist implements training exercises to learn how to handle freezing situations. GaitAssist is the result of a design process where we considered the input of engineers, clinicians and 18 Parkinsons disease patients, in order to find an optimal trade-off between system wearability and performance. We tested the final system in a user study with 5 additional patients. They reported a reduction in the freezing of gait duration as a result of the auditory stimulation provided, and that they feel the system enhanced their confidence during walking.

An Analysis of Sensor-Oriented vs. Model-Based Activity Recognition

Model-based activity recognition has been recently proposed as an alternative to signal-oriented recognition. Such model-based approaches seem attractive due to their ability to enable user-independent activity recognition and due to their improved robustness to signal-variation. The first goal of this paper is therefore to systematically analyze the benefit of body-model derived primitives in different sensor settings for multi activity recognition. Furthermore we propose a new body-model based approach using accelerometer sensors only thereby reducing the sensor requirements significantly. Results on a 20 activity dataset indicate that body-model based approaches consistently improve results over signal-oriented approaches.

GaitAssist: A wearable assistant for gait training and rehabilitation in Parkinson's disease

Many patients with Parkinsons disease suffer from short periods during which they cannot continue walking, the so-called freezing of gait. Patients can learn to use rhythmic auditory sounds as support during these episodes. We developed GaitAssist, a personalized wearable system for freezing of gait support, that enables training in unsupervised environments. GaitAssist detects freezing episodes from ankle-mounted motion sensors, which stream data via Bluetooth to an Android phone. In response, the system plays a rhythmic auditory sound that adapts to the patients regular gait speed. While GaitAssist can be used as a daily-life assistant, it also provides support for three types of training and rehabilitation exercises. The user can create personalized training sessions by adjusting the exercise and feedback parameters.

Smart crowds in smart cities: real life, city scale deployments of a smartphone based participatory crowd management platform

We describe a platform for smart, city-wide crowd management based on participatory mobile phone sensing and location/situation specific information delivery. The platform supports quick and flexible deployments of end-to-end applications for specific events or spaces that include four key functionalities: (1) Mobile phone based delivery of event/space specific information to the users, (2) participatory sensor data collection (from app users) and flexible analysis, (3) location and situation specific message multicast instructing people in different areas to act differently in case of an emergency and (4) post mortem event analysis. This paper describes the requirements that were derived through a series of test deployments, the system architecture, the implementation and the experiences made during real life, large scale deployments. Thus, until today it has been deployed at 14 events in three European countries (UK, Netherlands, Switzerland) and was used by well over 100,000 people.

A Wearable Assistant for Gait Training for Parkinson’s Disease with Freezing of Gait in Out-of-the-Lab Environments

People with Parkinson’s disease (PD) suffer from declining mobility capabilities, which cause a prevalent risk of falling. Commonly, short periods of motor blocks occur during walking, known as freezing of gait (FoG). To slow the progressive decline of motor abilities, people with PD usually undertake stationary motor-training exercises in the clinics or supervised by physiotherapists. We present a wearable system for the support of people with PD and FoG. The system is designed for independent use. It enables motor training and gait assistance at home and other unsupervised environments. The system consists of three components. First, FoG episodes are detected in real time using wearable inertial sensors and a smartphone as the processing unit. Second, a feedback mechanism triggers a rhythmic auditory signal to the user to alleviate freeze episodes in an assistive mode. Third, the smartphone-based application features support for training exercises. Moreover, the system allows unobtrusive and long-term monitoring of the user’s clinical condition by transmitting sensing data and statistics to a telemedicine service. We investigate the at-home acceptance of the wearable system in a study with nine PD subjects. Participants deployed and used the system on their own, without any clinical support, at their homes during three protocol sessions in 1 week. Users’ feedback suggests an overall positive attitude toward adopting and using the system in their daily life, indicating that the system supports them in improving their gait. Further, in a data-driven analysis with sensing data from five participants, we study whether there is an observable effect on the gait during use of the system. In three out of five subjects, we observed a decrease in FoG duration distributions over the protocol days during gait-training exercises. Moreover, sensing data-driven analysis shows a decrease in FoG duration and FoG number in four out of five participants when they use the system as a gait-assistive tool during normal daily life activities at home.

Engineers meet clinicians: augmenting Parkinson's disease patients to gather information for gait rehabilitation

Many people with Parkinsons disease suffer from freezing of gait, a debilitating temporary inability to pursue walking. Rehabilitation with wearable technology is promising. State of the art approaches face difficulties in providing the needed bio-feedback with a sufficient low-latency and high accuracy, as they rely solely on the crude analysis of movement patterns allowed by commercial motion sensors. Yet the medical literature hints at more sophisticated approaches. In this work we present our first step to address this with a rich multimodal approach combining physical and physiological sensors. We present the experimental recordings including 35 motion and 3 physiological sensors we conducted on 18 patients, collecting 23 hours of data. We provide best practices to ensure a robust data collection that considers real requirements for real world patients. To this end we show evidence from a user questionnaire that the system is low-invasive and that a multimodal view can leverage cross modal correlations for detection or even prediction of gait freeze episodes.