Kristina Yordanova

Computational state space models for activity and intention recognition. A feasibility study.

Background Computational state space models (CSSMs) enable the knowledge-based construction of Bayesian filters for recognizing intentions and reconstructing activities of human protagonists in application domains such as smart environments, assisted living, or security. Computational, i. e., algorithmic, representations allow the construction of increasingly complex human behaviour models. However, the symbolic models used in CSSMs potentially suffer from combinatorial explosion, rendering inference intractable outside of the limited experimental settings investigated in present research. The objective of this study was to obtain data on the feasibility of CSSM-based inference in domains of realistic complexity. Methods A typical instrumental activity of daily living was used as a trial scenario. As primary sensor modality, wearable inertial measurement units were employed. The results achievable by CSSM methods were evaluated by comparison with those obtained from established training-based methods (hidden Markov models, HMMs) using Wilcoxon signed rank tests. The influence of modeling factors on CSSM performance was analyzed via repeated measures analysis of variance. Results The symbolic domain model was found to have more than states, exceeding the complexity of models considered in previous research by at least three orders of magnitude. Nevertheless, if factors and procedures governing the inference process were suitably chosen, CSSMs outperformed HMMs. Specifically, inference methods used in previous studies (particle filters) were found to perform substantially inferior in comparison to a marginal filtering procedure. Conclusions Our results suggest that the combinatorial explosion caused by rich CSSM models does not inevitably lead to intractable inference or inferior performance. This means that the potential benefits of CSSM models (knowledge-based model construction, model reusability, reduced need for training data) are available without performance penalty. However, our results also show that research on CSSMs needs to consider sufficiently complex domains in order to understand the effects of design decisions such as choice of heuristics or inference procedure on performance.

Towards creating assistive software by employing human behavior models

Assistive software becomes more and more important part of our everyday life. As it is not straightforward to create such a system, the engineering of assistive systems is a topic of current research with different applications in healthcare, education and industry. In this paper we introduce three contributions to this field of research. Whereas most assistive systems use approaches for intention recognition based on training data applicable to specific environments and applications, we introduce a training-free approach. We do that by showing that it is possible to generate probabilistic inference systems from causal models for human behavior. Additionally, we collect a list of requirements for context aware assistive software and human behavior modeling for intention recognition and showed that our system satisfies them. We then introduce a software architecture for assistive systems that provides support for this kind of modeling. In addition to introducing the modeling approach and the architecture we show in an experimental way that our approach is suited for smart environments. The collected list of requirements could help a software engineer create a robust and easily adaptable to changes in the environment assistive software.

A Process for Systematic Development of Symbolic Models for Activity Recognition

Several emerging approaches to activity recognition (AR) combine symbolic representation of user actions with probabilistic elements for reasoning under uncertainty. These approaches provide promising results in terms of recognition performance, coping with the uncertainty of observations, and model size explosion when complex problems are modelled. But experience has shown that it is not always intuitive to model even seemingly simple problems. To date, there are no guidelines for developing such models. To address this problem, in this work we present a development process for building symbolic models that is based on experience acquired so far as well as on existing engineering and data analysis workflows. The proposed process is a first attempt at providing structured guidelines and practices for designing, modelling, and evaluating human behaviour in the form of symbolic models for AR. As an illustration of the process, a simple example from the office domain was developed. The process was evaluated in a comparative study of an intuitive process and the proposed process. The results showed a significant improvement over the intuitive process. Furthermore, the study participants reported greater ease of use and perceived effectiveness when following the proposed process. To evaluate the applicability of the process to more complex AR problems, it was applied to a problem from the kitchen domain. The results showed that following the proposed process yielded an average accuracy of 78%. The developed model outperformed state-of-the-art methods applied to the same dataset in previous work, and it performed comparably to a symbolic model developed by a model expert without following the proposed development process.

What's cooking and why? Behaviour recognition during unscripted cooking tasks for health monitoring

Nutrition related health conditions can seriously decrease quality of life; a system able to monitor the kitchen activities and eating behaviour of patients could provide clinicians with important indicators for improving a patients condition. To achieve this, the system has to reason about the persons actions and goals. To address this challenge, we present a behaviour recognition approach that relies on symbolic behaviour representation and probabilistic reasoning to recognise the persons actions, the type of meal being prepared and its potential impact on a patients health. We test our approach on a cooking dataset containing unscripted kitchen activities recorded with various sensors in a real kitchen. The results show that the approach is able to recognise the sequence of executed actions and the prepared meal, to determine whether it is healthy, and to reason about the possibility of depression based on the type of meal.

Marginal filtering in large state spaces

We describe the marginal filter for activity recognition using symbolic models.The marginal filter allows fine-grained activity recognition using wearable sensors.We identify and discuss advantages over particle filters for symbolic models. Recognising everyday activities including information about the context requires to handle large state spaces. The usage of wearable sensors like six degree of freedom accelerometers increases complexity even more. Common approaches are unable to maintain an accurate belief state within such complex domains. We show how marginal filtering can overcome limitations of standard particle filtering and efficiently infer the context of actions. Symbolic models of human behaviour are used to recognise activities in two different settings with different state space sizes. Based on these scenarios we compare the marginal filter to the standard particle filter. An evaluation shows that the marginal filter performs comparably in small state spaces but outperforms the particle filter in large state spaces.

Context aware approach for activity recognition based on precondition-effect rules

Context awareness plays an essential role in systems dealing with activity recognition. The context information present to the system, and the way in which it is modelled, shape the performance of the system during activity inference. In this paper we present a novel approach for modelling human behaviour based on preconditions and effects and employing it for generating training-free probabilistic models, that are later used for recognizing the user activities. Furthermore, we use our approach to recognize the activities in a three-person meeting and compare the results from our generated models with those of hand crafted and trained models. Finally, we show that we are able to successfully infer the user state even in models with huge state space.

Modelling Human Behaviour Using Partial Order Planning Based on Atomic Action Templates

A problem in assisting users in intelligent environments is the detection of their long term intentions based on the current state. One approach to solving this problem is the employment of human behaviour models, such as CTML and PDDL. At present, most of these models are based on concrete domains or scenarios which makes them difficult or impossible to adapt for other use cases. To overcome this drawback, this paper introduces an approach that uses partial order planning for generating a user behaviour model. Furthermore, it proposes a generalization of human activities by introducing atomic action templates valid for activities from various domains. To illustrate the approach, a scenario from the elderly care domain is modelled. Overall, the paper provides an effective general approach for modelling human behaviour which can be embedded in the intention inference workflow.

Towards a situation model for assessing challenging behaviour of people with dementia

With the increase of elderly population, the percentage of people suffering from dementia also increases. Typically, patients with dementia are cared for at home by family members. The task of caregiving is associated with significant psychological and physical stress that affects both the caregiver and the person with dementia. One solution to improving the task of caregiving is to provide an assistive system that is able to automatically recognise when challenging behaviour is exhibited and to provide suggestions for appropriate intervention strategies. To achieve that however, the system needs a situation model that provides the context information needed to recognise the type of behaviour and to reason about its causes. To address this problem, this paper performs a systematic analysis of the elements needed for building a situation model for assessing the behaviour of people with Alzheimers disease (AD). The analysis consists of literature review, interviews with experts, and brainstorming sessions. As a result, the work proposes a concept of a situation model for assessing the challenging behaviour of people with dementia based on sensor observations.

A Simple Model for Improving the Performance of the Stanford Parser for Action Detection in Textual Instructions.

Different approaches for behaviour understanding rely on textual instructions to generate models of human behaviour. These approaches usually use state of the art parsers to obtain the part of speech (POS) meaning and dependencies of the words in the instructions. For them it is essential that the parser is able to correctly annotate the instructions and especially the verbs as they describe the actions of the person. State of the art parsers usually make errors when annotating textual instructions, as they have short sentence structure often in imperative form. The inability of the parser to identify the verbs results in the inability of behaviour understanding systems to identify the relevant actions. To address this problem, we propose a simple rule-based model that attempts to correct any incorrectly annotated verbs. We argue that the model is able to significantly improve the parser’s performance without the need of additional training data. We evaluate our approach by extracting the actions from 61 textual instructions annotated only with the Stanford parser and once again after applying our model. The results show a significant improvement in the recognition rate when applying the rules (75% accuracy compared to 68% without the rules, p-value < 0.001).

Learning Models of Human Behaviour from Textual Instructions

There are various activity recognition approaches that rely on manual definition of precondition-effect rules to describe human behaviour. These rules are later used to generate computational models of human behaviour that are able to reason about the user behaviour based on sensor observations. One problem with these approaches is that the manual rule definition is time consuming and error prone process. To address this problem, in this paper we propose an approach that learns the rules from textual instructions. In difference to existing approaches, it is able to learn the causal relations between the actions without initial training phase. Furthermore, it learns the domain ontology that is used for the model generalisation and specialisation. To evaluate the approach, a model describing cooking task was learned and later applied for explaining seven plans of actual human behaviour. It was then compared to a hand-crafted model describing the same problem. The results showed that the learned model was able to recognise the plans with higher overall probability compared to the hand-crafted model. It also learned a more complex domain ontology and was more general than the hand-crafted model. In general, the results showed that it is possible to learn models of human behaviour from textual instructions which are able to explain actual human behaviour.