Britton Wolfe

Learning predictive state representations in dynamical systems without reset

Predictive state representations (PSRs) are a recently-developed way to model discrete-time, controlled dynamical systems. We present and describe two algorithms for learning a PSR model: a Monte Carlo algorithm and a temporal difference (TD) algorithm. Both of these algorithms can learn models for systems without requiring a reset action as was needed by the previously available general PSR-model learning algorithm. We present empirical results that compare our two algorithms and also compare their performance with that of existing algorithms, including an EM algorithm for learning POMDP models.

Comprehensive Behavior Profiling for Proactive Android Malware Detection

We present a new method of screening for malicious Android applications that uses two types of information about the application: the permissions that the application requests in its installation manifest and a metric called percentage of valid call sites (PVCS). PVCS measures the riskiness of the application based on a data flow graph. The information is used with machine learning algorithms to classify previously unseen applications as malicious or benign with a high degree of accuracy. Our classifier outperforms the previous state of the art by a significant margin, with particularly low false positive rates. Furthermore, the classifier evaluation is performed on malware families that were not used in the training phase, simulating the accuracy of the classifier on malware yet to be developed. We found that our PVCS metric and the SEND_SMS permission are the specific pieces of information that are most useful to the classifier.

High Precision Screening for Android Malware with Dimensionality Reduction

We present a new method of classifying previously unseen Android applications as malware or benign. The algorithm starts with a large set of features: the frequencies of all possible n-byte sequences in the applications byte code. Principal components analysis is applied to that frequency matrix in order to reduce it to a low-dimensional representation, which is then fed into any of several classification algorithms. We utilize the implicitly restarted Lanczos bidiagonalization algorithm and exploit the sparsity of the n-gram frequency matrix in order to efficiently compute the low-dimensional representation. When trained upon that low-dimensional representation, several classification algorithms achieve higher accuracy than previous work.

INCORPORATING PRIOR KNOWLEDGE INTO TEMPORAL DIFFERENCE NETWORKS

Developing general purpose algorithms for learning an accurate model of dynamical systems from example traces of the system is still a challenging researc h problem. Predictive State Representation (PSR) models represent the state of a dynamical system as a set of predictions about future events. Our work focuse s on improving Temporal Difference Networks (TD Nets), a general class of predictive state models. We adapt the internal structure of the TD Net and we present an improved algorithm for learning a TD Net model from experience in the environment. The new algorit hm accepts a set of known facts about the environme nt and uses those facts to accelerate the learning. Th ese facts can come from another learning algorithm (as in this study) or from a designer’s prior knowledge ab out the environment. Experiments demonstrate that using the new structure and learning algorithm impr oves the accuracy of the TD Net models. When tested in an in finite environment, our new algorithm outperf orms all of the standard PSR learning algorithms.

A Comparison of Algorithms for Handheld Wand Tracking

We examine several algorithms for tracking a handheld wand in a 3D virtual reality system: extended Kalman filters (EKFs), interacting multiple models (IMMs), and support vector machines (SVMs). The IMMs consist of several EKF models, each of which is tuned for one particular type of user motion. For determining the types of motion, we compare hand-created rules with an automatic clustering algorithm, with mixed results. The mode-specific EKFs within the IMM are more accurate than one overall EKF. However, the IMM is comparable to a single EKF, because of the overhead of predicting the current component EKF. SVMs with a one-frame lookahead perform the best, cutting the error in half. Aside from those SVMs, different model types were best for the different dimensions of tracking (x, y, z, and rotation).

Evaluating 3D Vision for Command and Control Applications

3D stereoscopic vision is used in many applications, but the level of benefit to the user differs depending on the particular application. We studied its benefits for command and control applications such as battlefield visualization or disaster response. We conducted experiments where the subjects completed some simple military planning exercises both with and without 3D vision. 3D users had lower error when judging line of sight between two points. Furthermore, survey results show that subjects preferred 3D. We also compared two ways of rendering symbols in the environment. Billboard symbols were more efficient than draping the symbol on the terrain.