Cuong Pham

Rapid specification and automated generation of prompting systems to assist people with dementia

Activity recognition in intelligent environments could play a key role for supporting people in their activities of daily life. Partially observable Markov decision process (POMDP) models have been used successfully, for example, to assist people with dementia when carrying out small multistep tasks such as hand washing. POMDP models are a powerful, yet flexible framework for modeling assistance that can deal with uncertainty and utility in a theoretically well-justified manner. Unfortunately, POMDPs usually require a very labor-intensive, manual set-up procedure. This paper describes a knowledge-driven method for automatically generating POMDP activity recognition and context-sensitive prompting systems for complex tasks. It starts with a psychologically justified description of the task and the particular environment in which it is to be carried out that can be generated from empirical data. This is then combined with a specification of the available sensors and effectors to build a working prompting system. The method is illustrated by building a system that prompts through the task of making a cup of tea in a real-world kitchen. The case is made that, with further development and tool support, the method could feasibly be used in a clinical or industrial setting.

Slice&Dice: Recognizing Food Preparation Activities Using Embedded Accelerometers

Within the context of an endeavor to provide situated support for people with cognitive impairments in the kitchen, we developed and evaluated classifiers for recognizing 11 actions involved in food preparation. Data was collected from 20 lay subjects using four specially designed kitchen utensils incorporating embedded 3-axis accelerometers. Subjects were asked to prepare a mixed salad in our laboratory-based instrumented kitchen environment. Video of each subjects food preparation activities were independently annotated by three different coders. Several classifiers were trained and tested using these features. With an overall accuracy of 82.9% our investigation demonstrated that a broad set of food preparation actions can be reliably recognized using sensors embedded in kitchen utensils.

Activity Recognition and Healthier Food Preparation

Obesity is an increasing problem for modern societies, which implies enormous financial burdens for public health-care systems. There is growing evidence that a lack of cooking and food preparation skills is a substantial barrier to healthier eating for a significant proportion of the population. We present the basis for a technological approach to promoting healthier eating by encouraging people to cook more often. We integrated tri-axial acceleration sensors into kitchen utensils (knifes, scoops, spoons), which allows us to continuously monitor the activities people perform while acting in the kitchen. A recognition framework is described, which discriminates ten typical kitchen activities. It is based on a sliding-window procedure that extracts statistical features for contiguous portions of the sensor data. These frames are fed into a Gaussian mixture density classifier, which provides recognition hypotheses in real-time. We evaluated the activity recognition system by means of practical experiments of unconstrained food preparation. The system achieves classification accuracy of ca. 90% for a dataset that covers 20 persons’ cooking activities.

A dynamic time warping approach to real-time activity recognition for food preparation

We present a dynamic time warping based activity recognition system for the analysis of low-level food preparation activities. Accelerometers embedded into kitchen utensils provide continuous sensor data streams while people are using them for cooking. The recognition framework analyzes frames of contiguous sensor readings in real-time with low latency. It thereby adapts to the idiosyncrasies of utensil use by automatically maintaining a template database. We demonstrate the effectiveness of the classification approach by a number of real-world practical experiments on a publically available dataset. The adaptive system shows superior performance compared to a static recognizer. Furthermore, we demonstrate the generalization capabilities of the system by gradually reducing the amount of training samples. The system achieves excellent classification results even if only a small number of training samples is available, which is especially relevant for real-world scenarios.

FoodBoard: surface contact imaging for food recognition

We describe FoodBoard, an instrumented chopping board that uses optical fibers and embedded camera imaging to identify unpackaged ingredients during food preparation on its surface. By embedding the sensing directly, and robustly, in the surface of a chopping board we also demonstrate how surface contact optical sensing can be used to realize the portability and privacy required of technology used in a setting such as a domestic kitchen. FoodBoard was subjected to a close to real-world evaluation in which 12 users prepared actual meals. FoodBoard compared favourably with existing unpackaged food recognition systems, classifying a larger number of distinct food ingredients (12 incl. meat, fruit, vegetables) with an average accuracy of 82.8%.

Distributed event processing for activity recognition

Stream-processing systems inevitably face unpredictable variations in incoming event loads. One way of handling this without affecting end-to-end performance metrics, will be to dynamically distribute event-processing on multiple computers and thus avail compute power for optimal performance. More precisely, data streams are processed in part or in parallel on multiple computers connected by a high bandwidth network. The number of computers being used is to be varied dynamically to cope with input load fluctuations. This paper uses data from ambient kitchen to make a preliminary assessment of performance advantages by distribution of real-time data stream processing. The motivation is to leverage cloud computing for optimal realtime event processing.