Thomas Plötz

Markov models for offline handwriting recognition: a survey

Since their first inception more than half a century ago, automatic reading systems have evolved substantially, thereby showing impressive performance on machine-printed text. The recognition of handwriting can, however, still be considered an open research problem due to its substantial variation in appearance. With the introduction of Markovian models to the field, a promising modeling and recognition paradigm was established for automatic offline handwriting recognition. However, so far, no standard procedures for building Markov-model-based recognizers could be established though trends toward unified approaches can be identified. It is therefore the goal of this survey to provide a comprehensive overview of the application of Markov models in the research field of offline handwriting recognition, covering both the widely used hidden Markov models and the less complex Markov-chain or n-gram models. First, we will introduce the typical architecture of a Markov-model-based offline handwriting recognition system and make the reader familiar with the essential theoretical concepts behind Markovian models. Then, we will give a thorough review of the solutions proposed in the literature for the open problems how to apply Markov-model-based approaches to automatic offline handwriting recognition.

Feature learning for activity recognition in ubiquitous computing

Feature extraction for activity recognition in context-aware ubiquitous computing applications is usually a heuristic process, informed by underlying domain knowledge. Relying on such explicit knowledge is problematic when aiming to generalize across different application domains. We investigate the potential of recent machine learning methods for discovering universal features for context-aware applications of activity recognition. We also describe an alternative data representation based on the empirical cumulative distribution function of the raw data, which effectively abstracts from absolute values. Experiments on accelerometer data from four publicly available activity recognition datasets demonstrate the significant potential of our approach to address both contemporary activity recognition tasks and next generation problems such as skill assessment and the detection of novel activities.

Multi-modal anchoring for human-robot interaction

Abstract This paper presents a hybrid approach for tracking humans with a mobile robot that integrates face and leg detection results extracted from image and laser range data, respectively. The different percepts are linked to their symbolic counterparts legs and face by anchors as defined by Coradeschi and Saffiotti [Anchoring symbols to sensor data: preliminary report, in: Proceedings of the Conference of the American Association for Artificial Intelligence, 2000, pp. 129–135]. In order to anchor the composite object person we extend the anchoring framework to combine different component anchors belonging to the same person. This allows to deal with perceptual algorithms having different spatio-temporal properties and provides a structured way for integrating anchor data from multiple modalities. An evaluation demonstrates the performance of our approach.

The mobile fitness coach: Towards individualized skill assessment using personalized mobile devices

We report on our extended research on GymSkill, a smartphone system for comprehensive physical exercising support, from sensor data logging, activity recognition to on-top skill assessment, using the phones built-in sensors. In two iterations, we used principal component breakdown analysis (PCBA) and criteria-based scores for individualized and personalized automated feedback on the phone, with the goal to track training quality and success and give feedback to the user, as well as to engage and motivate regular exercising. Qualitative feedback on the system was collected in a user study, and the system showed good evaluation results in an evaluation against manual expert assessments of video-recorded trainings.

Rapid specification and automated generation of prompting systems to assist people with dementia

Activity recognition in intelligent environments could play a key role for supporting people in their activities of daily life. Partially observable Markov decision process (POMDP) models have been used successfully, for example, to assist people with dementia when carrying out small multistep tasks such as hand washing. POMDP models are a powerful, yet flexible framework for modeling assistance that can deal with uncertainty and utility in a theoretically well-justified manner. Unfortunately, POMDPs usually require a very labor-intensive, manual set-up procedure. This paper describes a knowledge-driven method for automatically generating POMDP activity recognition and context-sensitive prompting systems for complex tasks. It starts with a psychologically justified description of the task and the particular environment in which it is to be carried out that can be generated from empirical data. This is then combined with a specification of the available sensors and effectors to build a working prompting system. The method is illustrated by building a system that prompts through the task of making a cup of tea in a real-world kitchen. The case is made that, with further development and tool support, the method could feasibly be used in a clinical or industrial setting.

ClimbAX: skill assessment for climbing enthusiasts

In recent years the sport of climbing has seen consistent increase in popularity. Climbing requires a complex skill set for successful and safe exercising. While elite climbers receive intensive expert coaching to refine this skill set, this progression approach is not viable for the amateur population. We have developed ClimbAX - a climbing performance analysis system that aims for replicating expert assessments and thus represents a first step towards an automatic coaching system for climbing enthusiasts. Through an accelerometer based wearable sensing platform, climbers movements are captured. An automatic analysis procedure detects climbing sessions and moves, which form the basis for subsequent performance assessment. The assessment parameters are derived from sports science literature and include: power, control, stability, speed. ClimbAX was evaluated in a large case study with 53 climbers under competition settings. We report a strong correlation between predicted scores and official competition results, which demonstrate the effectiveness of our automatic skill assessment system.

Physical activity, sedentary behaviour and metabolic control following stroke: a cross-sectional and longitudinal study.

Background Physical activity and sedentary behaviour are key moderators of cardiovascular disease risk and metabolic control. Despite the importance of a physically active lifestyle, little is known about the effects of stroke on physical activity. We assessed physical activity and sedentary behaviour at three time points following stroke compared to a healthy control group. Methods Physical activity and sedentary behaviour were objectively measured using a portable multi-sensor array in 31 stroke participants (73±9 years, National Institute of Health Stroke Scale 2±2, mobile 10 metres with/without aid) within seven days and at three and six months. Stroke data were compared with an age, sex and body mass index matched healthy control group (n = 31). Results Within seven days of stroke, total energy expenditure and physical activity were significantly lower and sedentary time higher in the stroke group compared to controls (total energy expenditure 1840±354 vs. 2220±489 kcal, physical activity 28±32 vs. 79±46 min/day, steps 3111±2290 vs. 7996±2649, sedentary time 1383±42 vs. 1339±44 min/day, p<0.01). At three months physical activity levels had increased (64±58 min/day) but plateaued by six months (66±68 min/day). Conclusions Physical activity levels are reduced immediately post-stroke and remain below recommended levels for health and wellbeing at the three and six month time points. Clinicians should explore methods to increase physical activity and reduce sedentary behaviour in both the acute and later stages following stroke.

Using unlabeled data in a sparse-coding framework for human activity recognition

We propose a sparse-coding framework for activity recognition in ubiquitous and mobile computing that alleviates two fundamental problems of current supervised learning approaches. (i) It automatically derives a compact, sparse and meaningful feature representation of sensor data that does not rely on prior expert knowledge and generalizes well across domain boundaries. (ii) It exploits unlabeled sample data for bootstrapping effective activity recognizers, i.e., substantially reduces the amount of ground truth annotation required for model estimation. Such unlabeled data is easy to obtain, e.g., through contemporary smartphones carried by users as they go about their everyday activities.Based on the self-taught learning paradigm we automatically derive an over-complete set of basis vectors from unlabeled data that captures inherent patterns present within activity data. Through projecting raw sensor data onto the feature space defined by such over-complete sets of basis vectors effective feature extraction is pursued. Given these learned feature representations, classification backends are then trained using small amounts of labeled training data.We study the new approach in detail using two datasets which differ in terms of the recognition tasks and sensor modalities. Primarily we focus on a transportation mode analysis task, a popular task in mobile-phone based sensing. The sparse-coding framework demonstrates better performance than the state-of-the-art in supervised learning approaches. More importantly, we show the practical potential of the new approach by successfully evaluating its generalization capabilities across both domain and sensor modalities by considering the popular Opportunity dataset. Our feature learning approach outperforms state-of-the-art approaches to analyzing activities of daily living.

Automatic assessment of problem behavior in individuals with developmental disabilities

Severe behavior problems of children with developmental disabilities often require intervention by specialists. These specialists rely on direct observation of the behavior, usually in a controlled clinical environment. In this paper, we present a technique for using on-body accelerometers to assist in automated classification of problem behavior during such direct observation. Using simulated data of episodes of severe behavior acted out by trained specialists, we demonstrate how machine learning techniques can be used to segment relevant behavioral episodes from a continuous sensor stream and to classify them into distinct categories of severe behavior (aggression, disruption, and self-injury). We further validate our approach by demonstrating it produces no false positives when applied to a publicly accessible dataset of activities of daily living. Finally, we show promising classification results when our sensing and analysis system is applied to data from a real assessment session conducted with a child exhibiting problem behaviors.

Large scale population assessment of physical activity using wrist worn accelerometers: The UK Biobank Study

Background Physical activity has not been objectively measured in prospective cohorts with sufficiently large numbers to reliably detect associations with multiple health outcomes. Technological advances now make this possible. We describe the methods used to collect and analyse accelerometer measured physical activity in over 100,000 participants of the UK Biobank study, and report variation by age, sex, day, time of day, and season. Methods Participants were approached by email to wear a wrist-worn accelerometer for seven days that was posted to them. Physical activity information was extracted from 100Hz raw triaxial acceleration data after calibration, removal of gravity and sensor noise, and identification of wear / non-wear episodes. We report age- and sex-specific wear-time compliance and accelerometer measured physical activity, overall and by hour-of-day, week-weekend day and season. Results 103,712 datasets were received (44.8% response), with a median wear-time of 6.9 days (IQR:6.5–7.0). 96,600 participants (93.3%) provided valid data for physical activity analyses. Vector magnitude, a proxy for overall physical activity, was 7.5% (2.35mg) lower per decade of age (Cohen’s d = 0.9). Women had a higher vector magnitude than men, apart from those aged 45-54yrs. There were major differences in vector magnitude by time of day (d = 0.66). Vector magnitude differences between week and weekend days (d = 0.12 for men, d = 0.09 for women) and between seasons (d = 0.27 for men, d = 0.15 for women) were small. Conclusions It is feasible to collect and analyse objective physical activity data in large studies. The summary measure of overall physical activity is lower in older participants and age-related differences in activity are most prominent in the afternoon and evening. This work lays the foundation for studies of physical activity and its health consequences. Our summary variables are part of the UK Biobank dataset and can be used by researchers as exposures, confounding factors or outcome variables in future analyses.