Christoph Burghardt

Towards creating assistive software by employing human behavior models

Assistive software becomes more and more important part of our everyday life. As it is not straightforward to create such a system, the engineering of assistive systems is a topic of current research with different applications in healthcare, education and industry. In this paper we introduce three contributions to this field of research. Whereas most assistive systems use approaches for intention recognition based on training data applicable to specific environments and applications, we introduce a training-free approach. We do that by showing that it is possible to generate probabilistic inference systems from causal models for human behavior. Additionally, we collect a list of requirements for context aware assistive software and human behavior modeling for intention recognition and showed that our system satisfies them. We then introduce a software architecture for assistive systems that provides support for this kind of modeling. In addition to introducing the modeling approach and the architecture we show in an experimental way that our approach is suited for smart environments. The collected list of requirements could help a software engineer create a robust and easily adaptable to changes in the environment assistive software.

Implementing Scenarios in a Smart Learning Environment

Among the scenarios for pervasive learning are teleteaching and project work. The former tries to deliver a lecture to a distributed audience, while the latter tries to enhance the learning effect by concentrated collaboration. In this paper, we want to show how an ambient environment, designed to support teams in a meeting, can be used to support pervasive learning scenarios as well as meetings. The developed applications help the users in both types of scenarios with pro-active assistance, enabling the users to focus on their task, rather than fiddling with the controls of the devices.

Supporting Ambient Environments by Extended Task Models

Task models are able to specify the behavior of actors in ambient environments in a compact and readable form. Their original notations have to be extended to meet the requirements in this context. The corresponding tool support has to be flexible in terms of integrating domain concepts rapidly. We propose a model-based approach for tool support of task model editors. By doing so, new modeling concepts can be introduced easily. Thus task models can be extended and provide a basis for analysis and high level design for a wide spread of domains. We fortify our approach by an example in the domain of AmI.

Inferring intentions in generic context-aware systems

Many context-aware projects try to develop the next step in human computer interaction, systems that adapt to a users need and help him to focus on his specific task. Probabilistic models are used to infer the current activity of a user. These techniques for predicting the actions of a user are often custom-tailored to a fixed location and scenario. We developed a method to generate probabilistic models for different context, therefore broaden their use in different domains of ubiquitous computing. This makes our intention analysis more generic.

Synthesising Generative Probabilistic Models for High-Level Activity Recognition

High-level (hierarchical) behaviour with long-term correlations is difficult to describe with first-order Markovian models like Hidden Markov models. We therefore discuss different approaches to synthesise generative probabilistic models for activity recognition based on different symbolic high-level description. Those descriptions of complex activities are compiled into robust generative models. The underlying assumptions for our work are (i) we need probabilistic models in robust activity recognition systems for the real world, (ii) those models should not necessarily rely on an extensive training phase and (iii) we should use available background knowledge to initialise them. We show how to construct such models based on different symbolic representations.

Making task modeling suitable for smart environments

The complicity of context information and task performance of actors in smart environment is demanding in terms of analysis, modeling and utilization. This paper proposes a task-based modeling approach, namely CTML, suitable for requirement stage of development to gain insight about the envisioned way tasks are performed. Additionally those models can be further employed to start-off the design phase of development which is also shown in the paper. Hence formal methods are employed to make CTML more amenable for software engineers due to its superior expressiveness.

Modelling Device Actions in Smart Environments

Smart environments are places that contain numerous devices to assist a user. Those devices’ actions can be modelled as planning operators. A problem when modelling such actions is the persistent action problem: Actions are not independent of one another. This is especially relevant when regarding persistent actions: An action that is being executed over a longer timespan may be terminated by a subsequent action that uses the same resources. The question is how to model this adequately. In dynamic environments with a high fluctuation of devices an additional challenge is to solve the persistent action problem with as little global information as possible. In this paper, we introduce two approaches: The first one locks resources which are being used by an action to prevent other actions from using the same resources. The second interleaves planning and execution of actions and is thus able to use software agents as “guards” for actions that are being executed. We furthermore compare the characteristics of both approaches and point out some implications those characteristics have on the modelling and execution of device actions in smart environments.

A Probabilistic Approach for Modeling Human Behavior in Smart Environments

In order to act intelligently, a smart environment needs to have a notion about its users. Hidden Markov models are especially suited to recognize for example the state of a meeting in a smart meeting room, as they can cope with the noisy and intermittent sensor values. However, modeling the user behavior as an HMM is challenging, because of the high degrees of freedom the users have when acting in such a smart environment. Therefore, we compare two methods that ease the automatic generation of HMM and express the human behavior.

Rapid Prototyping and Evaluation of Intention Analysis for Smart Environments

The development of smart environments is cumbersome and time-consuming compared to traditional software, since lacking a standard development process and according tool support. Smart environments are termed “smart” due to pro-active user assistance: User behavior is anticipated by an “intention analysis” software employing machine learning algorithms. In this paper we present a tool that facilitates the development of intention analysis by guiding the domain expert through the development process. Initially, the tool allows the user-centered design of HCI task models, without taking care of implementation details. Subsequently annotated task models are transformed into low level models, which are applied within the machine learning inference engine. We support both evaluation at early and later development stages. At early stages we evaluate designed models with expert-generated scenarios to simulate artificial low level sensor data. At later stages we evaluate a physical environment on the basis of real sensor data. A comparison between observed behavior and defined expectation allows identifying usability issues. A close connection between development and evaluation should further ensure rapid software changes and reevaluation to access improvements.