Michael Hardegger

GaitAssist: a daily-life support and training system for parkinson's disease patients with freezing of gait

Patients with Parkinsons disease often experience freezing of gait, which bears a high risk of falling, a prevalent cause for morbidity and mortality. In this work we present GaitAssist, a wearable system for freezing of gait support in daily life. The system provides real-time auditory cueing after the onset of freezing episodes. Furthermore, GaitAssist implements training exercises to learn how to handle freezing situations. GaitAssist is the result of a design process where we considered the input of engineers, clinicians and 18 Parkinsons disease patients, in order to find an optimal trade-off between system wearability and performance. We tested the final system in a user study with 5 additional patients. They reported a reduction in the freezing of gait duration as a result of the auditory stimulation provided, and that they feel the system enhanced their confidence during walking.

ActionSLAM: Using location-related actions as landmarks in pedestrian SLAM

Indoor localization at minimal deployment effort and with low costs is relevant for many ambient intelligence and mobile computing applications. This paper presents ActionSLAM, a novel approach to Simultaneous Localization And Mapping (SLAM) for pedestrian indoor tracking that makes use of body-mounted sensors. ActionSLAM iteratively builds a map of the environment and localizes the user within this map. A foot-mounted Inertial Measurement Unit (IMU) keeps track of the users path, while observations of location-related actions (e.g. door-opening or sitting on a chair) are used to compensate for drift error accumulation in a particle filter framework. Location-related actions are recognizable from body-mounted IMUs that are often used in ambient-assisted living scenarios for context awareness. Thus localization relies only on on-body sensing and requires no ambient infrastructure such as Wi-Fi access points or radio beacons. We characterize ActionSLAM on a dataset of 1.69km walking in three rooms and involving 241 location-related actions. For the experimental dataset, the algorithm robustly tracked the subject with mean error of 1.2m. The simultaneously built map reflects the building layout and positions landmarks with a mean error of 0.5m. These results were achieved with a simulated action recognition system consisting of an IMU attached to the wrist of a user and a smartphone in his pocket. We found that employing more complex action recognition is not beneficial for ActionSLAM performance. Our findings are supported by evaluations in synthetic environments through simulation of IMU signals for walks in typical home scenarios.

Improved actionSLAM for long-term indoor tracking with wearable motion sensors

We present an indoor tracking system based on two wearable inertial measurement units for tracking in home and workplace environments. It applies simultaneous localization and mapping with user actions as landmarks, themselves recognized by the wearable sensors. The approach is thus fully wearable and no pre-deployment effort is required. We identify weaknesses of past approaches and address them by introducing heading drift compensation, stance detection adaptation, and ellipse landmarks. Furthermore, we present an environment-independent parameter set that allows for robust tracking in daily-life scenarios. We assess the method on a dataset with five participants in different home and office environments, totaling 8.7h of daily routines and 2500m of travelled distance. This dataset is publicly released. The main outcome is that our algorithm converges 87% of the time to an accurate approximation of the ground truth map (0.52m mean landmark positioning error) in scenarios where previous approaches fail.

A Mobile Kalman-Filter Based Solution for the Real-Time Estimation of Spatio-Temporal Gait Parameters

Gait impairments are among the most disabling symptoms in several musculoskeletal and neurological conditions, severely limiting personal autonomy. Wearable gait sensors have been attracting attention as diagnostic tool for gait and are emerging as promising tool for tutoring and guiding gait execution. If their popularity is continuously growing, still there is room for improvement, especially towards more accurate solutions for spatio-temporal gait parameters estimation. We present an implementation of a zero-velocity-update gait analysis system based on a Kalman filter and off-the-shelf shoe-worn inertial sensors. The algorithms for gait events and step length estimation were specifically designed to comply with pathological gait patterns. More so, an Android app was deployed to support fully wearable and stand-alone real-time gait analysis. Twelve healthy subjects were enrolled to preliminarily tune the algorithms; afterwards sixteen persons with Parkinsons disease were enrolled for a validation study. Over the 1314 strides collected on patients at three different speeds, the total root mean square difference on step length estimation between this system and a gold standard was 2.9%. This shows that the proposed method allows for an accurate gait analysis and paves the way to a new generation of mobile devices usable anywhere for monitoring and intervention.

ActionSLAM on a smartphone: At-home tracking with a fully wearable system

We present SmartActionSLAM, an Android smartphone application that performs location tracking in home and office environments. It uses the integrated motion sensors of the smartphone and an optional foot-mounted inertial measurement unit to track a person. The application implements an instance of the ActionSLAM algorithmic framework. ActionSLAM combines pedestrian dead reckoning with the observation of activities (in SmartActionSLAM: sitting and standing still) to build and update a local landmark map of the users environment. This map is used to compensate for error accumulation of dead reckoning in a particle filter framework. We show that it is possible to execute the ActionSLAM algorithm in real-time on a smartphone without platform-specific optimizations. Furthermore, we analyze the localization performance of the application in six constrained and two real-life recordings. When using only the smartphones internal sensors, tracking was adequate in most constrained setups, but failed in the real-world scenarios because of errors in recognizing irregular leg movements. By including the foot-mounted sensor, mapping with a mean landmark positioning error of <; 0.5m and robustness > 90% was achieved in all environments. Smart-ActionSLAM is fully wearable and requires no infrastructure in the environment. The approach is therefore ideally suited for rapid deployment in home and office environments, as for example required in patient monitoring studies.

GaitAssist: A wearable assistant for gait training and rehabilitation in Parkinson's disease

Many patients with Parkinsons disease suffer from short periods during which they cannot continue walking, the so-called freezing of gait. Patients can learn to use rhythmic auditory sounds as support during these episodes. We developed GaitAssist, a personalized wearable system for freezing of gait support, that enables training in unsupervised environments. GaitAssist detects freezing episodes from ankle-mounted motion sensors, which stream data via Bluetooth to an Android phone. In response, the system plays a rhythmic auditory sound that adapts to the patients regular gait speed. While GaitAssist can be used as a daily-life assistant, it also provides support for three types of training and rehabilitation exercises. The user can create personalized training sessions by adjusting the exercise and feedback parameters.

S-SMART: A Unified Bayesian Framework for Simultaneous Semantic Mapping, Activity Recognition, and Tracking

The machine recognition of user trajectories and activities is fundamental to devise context-aware applications for support and monitoring in daily life. So far, tracking and activity recognition were mostly considered as orthogonal problems, which limits the richness of possible context inference. In this work, we introduce the novel unified computational and representational framework S-SMART that simultaneously models the environment state (semantic mapping), localizes the user within this map (tracking), and recognizes interactions with the environment (activity recognition). Thus, S-SMART identifies which activities the user executes where (e.g., turning a handle next to a window), and reflects the outcome of these actions by updating the world model (e.g., the window is now open). This in turn conditions the future possibility of executing actions at specific places (e.g., closing the window is likely to be the next action at this location). S-SMART works in a self-contained manner and iteratively builds the semantic map from wearable sensors only. This enables the seamless deployment to new environments. We characterize S-SMART in an experimental dataset with people performing hand actions as part of their usual routines at home and in office buildings. The framework combines dead reckoning from a foot-worn motion sensor with template-matching-based action recognition, identifying objects in the environment (windows, doors, water taps, phones, etc.) and tracking their state (open/closed, etc.). In real-life recordings with up to 23 action classes, S-SMART consistently outperforms independent systems for positioning and activity recognition, and constructs accurate semantic maps. This environment representation enables novel applications that build upon information about the arrangement and state of the user’s surroundings. For example, it may be possible to remind elderly people of a window that they left open before leaving the house, or of a plant they did not water yet, using solely wearable sensors.

Enhancing action recognition through simultaneous semantic mapping from body-worn motion sensors

Locations and actions are interrelated: some activities tend to occur at specific places, for example a person is more likely to twist his wrist when he is close to a door (to turn the knob). We present an unsupervised fusion method that takes advantage of this characteristic to enhance the recognition of location-related actions (e.g., open, close, switch, etc.). The proposed LocAFusion algorithm acts as a post-processing filter: At run-time, it constructs a semantic map of the environment by tagging action recognitions to Cartesian coordinates. It then uses the accumulated information about a location i) to discriminate between identical actions performed at different places and ii) to correct recognitions that are unlikely, given the other observations at the same location. LocAFusion does not require prior statistics about where activities occur, which allows for seamless deployment to new environments. The fusion approach is agnostic to the sensor modalities and methods used for action recognition and localization. For evaluation, we implemented a fully wearable setup that tracks the user with a foot-mounted motion sensor and the ActionSLAM algorithm. Simultaneously, we recognize hand actions through template matching on the data of a wrist-worn inertial measurement unit. In 10 recordings with 554 performed object interactions, LocAFusion consistently outperformed location-independent action recognition (8--31% increase in F1 score), identified 96% of the objects in the semantic map and overall correctly labeled 82% of the actions in problems with up to 23 classes.

Sensor technology for ice hockey and skating

Sensor technology that is unobtrusively integrated into the clothing and equipment of an athlete can support the training of sport activities and monitor the athletes progress. In this paper, we propose two wearable systems that support ice hockey players in the training of skating and shooting. These assistants measure the motions of players and compare them with reference executions of the same activities by professional players. A third system that we introduce monitors the player;s activities during a hockey game and creates a match report for objective performance measurement. For each of the three proposed applications, we present a prototype setup that we evaluate with amateur and professional players. The main findings are i) that with a skate-worn motion sensor and user-dependent training, eight skating motions can be spotted with an accuracy above 90%, ii) that stick-integrated sensors enable the measurement of relevant shot features, which differentiate professional from amateur athletes, and iii) that it is possible to spot important ice hockey activities in the signals of body-worn motion sensors worn during a game.

RFID-die: battery-free orientation sensing using an array of passive tilt switches

We here demonstrate the combination of tilt switches with RFID for building fully passive orientation-sensitive devices. We show the functionality of this approach with a simple die that can be read out with an NFC-enabled smartphone. As the proposed orientation system does not require battery powering, it is of interest to various wearable-computing and smart-home applications that benefit from long system runtime.