Ghim-Eng Yap

Discovering and Exploiting Causal Dependencies for Robust Mobile Context-Aware Recommenders

Acquisition of context poses unique challenges to mobile context-aware recommender systems. The limited resources in these systems make minimizing their context acquisition a practical need, and the uncertainty in the mobile environment makes missing and erroneous context inputs a major concern. In this paper, we propose an approach based on Bayesian networks (BNs) for building recommender systems that minimize context acquisition. Our learning approach iteratively trims the BN-based context model until it contains only the minimal set of context parameters that are important to a user. In addition, we show that a two-tiered context model can effectively capture the causal dependencies among context parameters, enabling a recommender system to compensate for missing and erroneous context inputs. We have validated our proposed techniques on a restaurant recommendation data set and a Web page recommendation data set. In both benchmark problems, the minimal sets of context can be reliably discovered for the specific users. Furthermore, the learned Bayesian network consistently outperforms the J4.8 decision tree in overcoming both missing and erroneous context inputs to generate significantly more accurate predictions.

Dynamically-optimized context in recommender systems

Traditional approaches to recommender systems have not taken into account situational information when making recommendations, and this seriously limits the relevance of the results. This paper advocates context-awareness as a promising approach to enhance the performance of recommenders, and introduces a mechanism to realize this approach. We present a framework that separates the contextual concerns from the actual recommendation module, so that contexts can be readily shared across applications. More importantly, we devise a learning algorithm to dynamically identify the optimal set of contexts for a specific recommendation task and user. An extensive series of experiments has validated that our system is indeed able to learn both quickly and accurately.

Explaining inferences in Bayesian networks

Abstract While Bayesian network (BN) can achieve accurate predictions even with erroneous or incomplete evidence, explaining the inferences remains a challenge. Existing approaches fall short because they do not exploit variable interactions and cannot account for compensations during inferences. This paper proposes the Explaining BN Inferences (EBI) procedure for explaining how variables interact to reach conclusions. EBI explains the value of a target node in terms of the influential nodes in the target’s Markov blanket under specific contexts, where the Markov nodes include the target’s parents, children, and the children’s other parents. Working back from the target node, EBI shows the derivation of each intermediate variable, and finally explains how missing and erroneous evidence values are compensated. We validated EBI on a variety of problem domains, including mushroom classification, water purification and web page recommendation. The experiments show that EBI generates high quality, concise and comprehensible explanations for BN inferences, in particular the underlying compensation mechanism that enables BN to outperform alternative prediction systems, such as decision tree.

Improving the accuracy of erroneous-plan recognition system for Activities of Daily Living

Using ambient intelligence to assist people with dementia in carrying out their Activities of Daily Living (ADLs) independently in smart home environment is an important research area, due to the projected increasing number of people with dementia. We present herein, a system and algorithms for the automated recognition of ADLs; the ADLs are in terms of plans made up encoded sequences of micro-context information gathered by sensors in a smart home. Previously, the Erroneous-Plan Recognition (EPR) system was developed to specifically handle the wide spectrum of micro contexts from multiple sensing modalities. The EPR system monitors the person with dementia and determines if he has executed a correct or erroneous ADL. However, due to the noisy readings of the sensing modalities, the EPR system has problems in accurately detecting the erroneous ADLs. We propose to improve the accuracy of the EPR system by two new key components. First, we model the smart home environment as a Markov decision process (MDP), with the EPR system built upon it. Simple referencing of this model allows us to filter erroneous readings of the sensing modalities. Second, we use the reinforcement learning concept of probability and reward to infer erroneous readings that are not filtered by the first key component.We conducted extensive experiments and showed that the accuracy of the new EPR system is 26.2% higher than the previous system, and is therefore a better system for ambient assistive living applications.

Centroid-Based Actionable 3D Subspace Clustering

Actionable 3D subspace clustering from real-world continuous-valued 3D (i.e., object-attribute-context) data promises tangible benefits such as discovery of biologically significant protein residues and profitable stocks, but existing algorithms are inadequate in solving this clustering problem; most of them are not actionable (ability to suggest profitable or beneficial actions to users), do not allow incorporation of domain knowledge, and are parameter sensitive, i.e., the wrong threshold setting reduces the cluster quality. Moreover, its 3D structure complicates this clustering problem. We propose a centroid-based actionable 3D subspace clustering framework, named CATSeeker, which allows incorporation of domain knowledge, and achieves parameter insensitivity and excellent performance through a unique combination of singular value decomposition, numerical optimization, and 3D frequent itemset mining. Experimental results on synthetic, protein structural, and financial data show that CATSeeker significantly outperforms all the competing methods in terms of efficiency, parameter insensitivity, and cluster usefulness.

Discovering Causal Dependencies in Mobile Context-Aware Recommenders

Mobile context-aware recommender systems face unique challenges in acquiring context. Resource limitations make minimizing context acquisition a practical need, while the uncertainty inherent to the mobile environment makes missing context values a major concern. This paper introduces a scalable mechanism based on Bayesian network learning in a tiered context model to overcome both of these challenges. Extensive experiments on a restaurant recommender system showed that our mechanism can accurately discover causal dependencies among context, thereby enabling the effective identification of the minimal set of important context for a specific user and task, as well as providing highly accurate recommendations even when context values are missing.

A Bayesian approach integrating regional and global features for image semantic learning

In content-based image retrieval, the “semantic gap” between visual image features and user semantics makes it hard to predict abstract image categories from low-level features. We present a hybrid system that integrates global features (G-features) and region features (R-features) for predicting image semantics. As an intermediary between image features and categories, we introduce the notion of mid-level concepts, which enables us to predict an images category in three steps. First, a G-prediction system uses G-features to predict the probability of each category for an image. Simultaneously, a R-prediction system analyzes R-features to identify the probabilities of mid-level concepts in that image. Finally, our hybrid H-prediction system based on a Bayesian network reconciles the predictions from both R-prediction and G-prediction to produce the final classifications. Results of experimental validations show that this hybrid system outperforms both G-prediction and R-prediction significantly.

Activity recognition using correlated pattern mining for people with dementia

Due to the rapidly aging population around the world, senile dementia is growing into a prominent problem in many societies. To monitor the elderly dementia patients so as to assist them in carrying out their basic Activities of Daily Living (ADLs) independently, sensors are deployed in their homes. The sensors generate a stream of context information, i.e., snippets of the patients current happenings, and pattern mining techniques can be applied to recognize the patients activities based on these micro contexts. Most mining techniques aim to discover frequent patterns that correspond to certain activities. However, frequent patterns can be poor representations of activities. In this paper, instead of using frequent patterns, we propose using correlated patterns to represent activities. Using simulation data collected in a smart home testbed, our experimental results show that using correlated patterns rather than frequent ones improves the recognition performance by 35.5% on average.

CDR-To-MoVis: Developing a Mobility Visualization System from CDR data

CDR (Call Detail Records) data is more easily available than other network related data (such as GPS data) as most telecommunications service providers (TSPs) maintain such data. By analyzing it one can find mobility patterns of most of the population thus leading to efficient urban planning, disease and traffic control, etc. But its granularity is low as the latitude and longitude (lat-lon) of a cell tower is used as the current location of all mobile phones that are connected to the cell tower at that time. Granularity can range between 10s of metres to several kms depending on population density of a locality. This is one reason why, although there are many existing systems on visualizing mobility of people based on GPS data, there is hardly any existing system for CDR. We develop a Mobility Visualization System (MoVis) for visualizing mobility of people from their CDR records. First of all, given the CDR records of a user, we determine her stay regions (places where she stays for a significant amount of time). Trajectories of phone events (and lat-lon of cell towers) between stay regions are extracted as her trips. Start and end times of a trip are estimated using linear extrapolation. Based on the start and end times, temporal patterns are extracted. Trips with sufficient number of intermediate points are mapped to transport network that consists of train lines, bus routes and expressways. We use Kernel density estimation to visualize the most common path for a given origin and destination. Based on this we create a round-the-clock visualization of mobility of people over the entire city separately for weekdays and weekends. At the end we show the validation results.

Learning Feature Dependencies for Noise Correction in Biomedical Prediction

The presence of noise or errors in the stated feature values of biomedical data can lead to incorrect prediction. We introduce a Bayesian Network-based Noise Correction framework named BN-NC. After data preprocessing, a Bayesian Network (BN) is learned to capture the feature dependencies. Using the BN to predict each feature in turn, BN-NC estimates a feature’s error rate as the deviation between its predicted and stated values in the training data, and allocates the appropriate uncertainty to its subsequent findings during prediction. BN-NC automatically generates a probabilistic rule to explain BN prediction on the class variable using the feature values in its Markov blanket, and this is reapplied as necessary to explain the noise correction on those features. Using three real-life benchmark biomedical data sets (on HIV-1 drug resistance prediction and leukemia subtype classification), we demonstrate that BN-NC (1) accurately detects the errors in biomedical feature values, (2) automatically corrects for the errors to maintain higher prediction accuracy over competing methods including Decision Trees, Naive Bayes and Support Vector Machines, and (3) generates probabilistic rules that concisely explain the prediction and noise correction decisions. In addition to achieving more robust biomedical prediction in the presence of feature noise, by highlighting erroneous features and explaining their corrections, BN-NC provides medical researchers with high utility insights to biomedical data not found in other methods.