Jayachandran Maniyeri

Processing of wearable sensor data on the cloud - a step towards scaling of continuous monitoring of health and well-being

As part of a sleep monitoring project, we used actigraphy based on body-worn accelerometer sensors to remotely monitor and study the sleep-wake cycle of elderly staying at nursing homes. We have conducted a fifteen patient trial of a sleep activity pattern monitoring (SAPM) system at a local nursing home. The data was collected and stored in our server and the processing of the data was done offline after sleep diaries used for validation and ground truth were updated into the system. The processing algorithm matches and annotates the sensor data with manual sleep diary information and is processed asynchronously on the grid/cloud back end. In this paper we outline the mapping of the system for grid / cloud processing, and initial results that show expected near-linear performance for scaling the number of users.

2-layer Erroneous-Plan Recognition for dementia patients in smart homes

People with dementia lose their ability to learn, solve problems, and communicate. And they are all around us. To potentially replace some of their diminished memory and problem-solving abilities, Erroneous-Plan Recognition (EPR) aims to detect defects or faults in the execution of correct plans by the dementia patient, and send timely audio and visual prompts to the dementia patient and caregiver in order to correct these faults. The scope of this work is for the patient who lives alone in a smart home. One challenge is that the definition of plan can be very subjective. It is necessary to regard a plan as an Activity of Daily Living (ADL), choose the ADLs to monitor, and deploy available sensors to acquire data. With the sensor data, there can be activity recognition, followed by plan recognition. Another challenge is the highly random and erroneous behaviour of dementia patients. Multiple, sequential, and independent layers of error detection can be arranged in a prioritised manner to detect specific errors first, and provide an error probability if no specific errors are detected. On the whole, most of the EPR results are very good as they are at least 0.9, indicating that the data is linearly separable. The 2-layer EPR system, which uses the blacklist and whitelist as Layer 1 and naive Bayes classifier as Layer 2, is significantly more accurate than each individual layer. In fact, 5 out of 6 actors have an accuracy above 0.9. With the encouraging results, there will be more technical and domain challenges which we can address in the near future.

Smart mote-based medical system for monitoring and handling medication among persons with dementia

This paper presents a novel smart mote-based portable medical system which automatically monitors and handles medication among persons with dementia based on wireless multimodal sensors, actuators and mobile phone or PDA (Personal Digital Assistance) technology. In particular, we present the subtle design, implementation and deployment issues of monitoring the patients behavior and providing adaptive assistive intervention such as prompts or reminders in the form of visual, audio or text cues to the patient for medical compliance. In addition, we develop mobile phone or PDA applications to provide a number of novel services to the caregivers that facilitate them in care-giving and to doctors for clinical assessment of dementia patients in a context enlightened fashion.

Heart Rate Estimation from FBG Sensors using Cepstrum Analysis and Sensor Fusion

This paper presents a method of estimating heart rate from arrays of fiber Bragg grating (FBG) sensors embedded in a mat. A cepstral domain signal analysis technique is proposed to characterize Ballistocardiogram (BCG) signals. With this technique, the average heart beat intervals can be estimated by detecting the dominant peaks in the cepstrum, and the signals of multiple sensors can be fused together to obtain higher signal to noise ratio than each individual sensor. Experiments were conducted with 10 human subjects lying on 2 different postures on a bed. The estimated heart rate from BCG was compared with heart rate ground truth from ECG, and the mean error of estimation obtained is below 1 beat per minute (BPM). The results show that the proposed fusion method can achieve promising heart rate measurement accuracy and robustness against various sensor contact conditions.

Fast matching of sensor data with manual observations

In systems and trials concerning wearable sensors and devices used for medical data collection, the validation of sensor data with respect to manual observations is very important. However, this is often problematic because of feigned behavior, errors in manual recording (misclassification), gaps in recording (missing readings), missed observations and timing mismatch between manual observations and sensor data due to a difference in time granularity. Using sleep activity pattern monitoring as an example we present a fast algorithm for matching sensor data with manual observations. Major components include a) signal analysis to classify states of sleep activity pattern, b) matching of states with Sleep Diary (SD) and c) automated detection of anomalies and reconciliation of mismatches between the SD and the sensor data.

Multimodal information fusion for automated recognition of complex agitation behaviors of dementia patients

This paper presents a new approach using multimodal information fusion for automated recognition of complex agitation behavior among persons with dementia. In particular, we present a hierarchical information fusion framework to model complex agitation behaviors based on a clinical agitation scale widely used in hospitals. We present the detailed features extraction and selection to represent low level atomic agitation behaviors, classifiers for atomic agitation behavior recognition and inference combiners to detect and rate the onset of complex agitation behaviors in dementia patients using multimodal sensors such as pressure sensors, ultrasound sensors, video cameras, acoustic sensors, etc., in a hospital ward.

Service Oriented Architecture for Patient Monitoring Application

This paper presents a new approach that exploits service oriented architecture (SOA) for patient monitoring application. In particular, we describe the use of web services, UPnP and semantic web in a loosely coupled manner to monitor dementia patients, model sensors and pervasive devices as services, and enable proactive action for handling patients by automatically triggering intervention services. The proposed architecture enables the development of a sophisticated monitoring system to facilitate care-giving and clinical assessment of patients in a context enlightened fashion within an administrative domain and across the internet that exceeds the current state-of-the-art in terms of scalability, interoperability, intelligence and robustness.

Detection of variations in cognitive workload using multi-modality physiological sensors and a large margin unbiased regression machine

Physiological sensor based workload estimation technology provides a real-time means for assessing cognitive workload and has a broad range of applications in cognitive ergonomics, mental health monitoring, etc. In this paper we report a study on detecting changes in workload using multi-modality physiological sensors and a novel feature extraction and classification algorithm. We conducted a cognitive workload experiment involving multiple subjects and collected an extensive data set of EEG, ECG and GSR signals. We show that the GSR signal is consistent with the variations of cognitive workload in 75% of the samples. To explore cardiac patterns in ECG that are potentially correlated with the cognitive workload process, we computed various heart-rate-variability features. To extract neuronal activity patterns in EEG related to cognitive workload, we introduced a filter bank common spatial pattern filtering technique. As there can be large variations in e.g. individual responses to the cognitive workload, we propose a large margin unbiased recursive feature extraction and regression method. Our leave-one-subject-out cross validation test shows that, using the proposed method, EEG can provide significantly better prediction of the cognitive workload variation than ECG, with 87.5% vs 62.5% in accuracy rate.

From Context to Micro-context – Issues and Challenges in Sensorizing Smart Spaces for Assistive Living

Most smart home based monitoring / assistive systems that attempt to recognize activities within a smart home are targeted towards living-alone elderly, and stop at providing instantaneous coarse grained information such as room-occupancy or provide specific programmed reminders for taking medication etc. In our work, we target multiple residents, while restricting the use of wearable devices / sensors. In addition we do away with video due to privacy concerns. In this paper we present the design challenges and issues in putting together a sensor network for obtaining micro-context information in multi-person smart spaces. In order to support greater levels of ambient intelligence we support fine grained spatio-temporal data and context acquisition. The architecture is being currently developed into a prototype in a modular fashion for deployment and testing in a variety of environments, and is being concurrently evaluated and tested in real conditions, prior to deployment in a facility for elderly residents with mild cognitive disorder.

Sensor data quality processing for vital signs with opportunistic ambient sensing

Opportunistic ambient sensing involves placement of sensors appropriately so that intermittent contact can be made unobtrusively for gathering physiological signals for vital signs. In this paper, we discuss the results of our quality processing system used to extract heart rate from ballisto-cardiogram signals obtained from a micro-bending fiber optic sensor pressure mat. Visual inspection is used to label data into informative and non-informative classes based on their heart rate information. Five classifiers are employed for the classification process, i.e., random forest, support vector machine, multilayer, feedforward neural network, linear discriminant analysis, and decision tree. To compute the overall effectiveness of quality processing, the informative signals are processed to estimate interbeat intervals. The system was used to process, data collected from 50 human subjects sitting in a massage chair while performing different activities. Opportunistically collected data was obtained from the fiber optic sensor mat placed on the headrest of the massage chair. Using our classification approach, 57.37% of the dataset was able to provide informative signals. On the informative signals, random forest classifier achieves the best classification accuracy with a mean accuracy of 98.99%. The average of the mean absolute error between the estimated heart rate and the reference ECG is reduced from 13.2 to 8.47. Therefore, the proposed system shows a good robustness for opportunistic ambient sensing.