Wesley Tansey

Annotation refactoring: inferring upgrade transformations for legacy applications

Since annotations were added to the Java language, many frameworks have moved to using annotated Plain Old Java Objects (POJOs) in their newest releases. Legacy applications are thus forced to undergo extensive restructuring in order to migrate from old framework versions to new versions based on annotations (Version Lock-in). Additionally, because annotations are embedded in the application code, changing between framework vendors may also entail largescale manual changes (Vendor Lock-in). This paper presents a novel refactoring approach that effectively solves these two problems. Our approach infers a concise set of semantics-preserving transformation rules from two versions of a single class. Unlike prior approaches that detect only simple structural refactorings, our algorithm can infer general composite refactorings and is more than 97% accurate on average. We demonstrate the effectiveness of our approach by automatically upgrading more than 80K lines of the unit testing code of four open-source Java applications to use the latest version of the popular JUnit testing framework.

Efficient automated marshaling of C++ data structures for MPI applications

We present an automated approach for marshaling C++ data structures in high performance computing (HPC) applications. Our approach utilizes a graphical editor through which the user can express a subset of an objects state to be marshaled and sent across a network. Our tool, MPI serializer, then automatically generates efficient marshaling and unmarshaling code for use with the message passing interface (MPI), the predominant communication middleware for HPC systems. Our approach provides a more comprehensive level of support for C++ language features than the existing state of the art, and does so in full compliance with the C++ language standard. Specifically, we can marshal effectively and efficiently non-trivial language constructs such as polymorphic pointers, dynamically allocated arrays, non-public member fields, inherited members, and STL container classes. Additionally, our marshaling approach is also applicable to third party libraries, as it does not require any modifications to the existing C++ source code. We validate our approach through two case studies of applying our tool to automatically generate the marshaling functionality of two realistic HPC applications. The case studies demonstrate that the automatically generated code matches the performance of typical hand-written implementations and surpasses current state-of-the-art C++ marshaling libraries, in some cases by more than an order of magnitude. The results of our case studies indicate that our approach can be beneficial for both the initial construction of HPC applications as well as for the refactoring of sequential applications for parallel execution.

False discovery rate smoothing

ABSTRACT We present false discovery rate (FDR) smoothing, an empirical-Bayes method for exploiting spatial structure in large multiple-testing problems. FDR smoothing automatically finds spatially localized regions of significant test statistics. It then relaxes the threshold of statistical significance within these regions, and tightens it elsewhere, in a manner that controls the overall false discovery rate at a given level. This results in increased power and cleaner spatial separation of signals from noise. The approach requires solving a nonstandard high-dimensional optimization problem, for which an efficient augmented-Lagrangian algorithm is presented. In simulation studies, FDR smoothing exhibits state-of-the-art performance at modest computational cost. In particular, it is shown to be far more robust than existing methods for spatially dependent multiple testing. We also apply the method to a dataset from an fMRI experiment on spatial working memory, where it detects patterns that are much more biologically plausible than those detected by standard FDR-controlling methods. All code for FDR smoothing is publicly available in Python and R (https://github.com/tansey/smoothfdr). Supplementary materials for this article are available online.

Multiscale spatial density smoothing: an application to large-scale radiological survey and anomaly detection

ABSTRACT We consider the problem of estimating a spatially varying density function, motivated by problems that arise in large-scale radiological survey and anomaly detection. In this context, the density functions to be estimated are the background gamma-ray energy spectra at sites spread across a large geographical area, such as nuclear production and waste-storage sites, military bases, medical facilities, university campuses, or the downtown of a city. Several challenges combine to make this a difficult problem. First, the spectral density at any given spatial location may have both smooth and nonsmooth features. Second, the spatial correlation in these density functions is neither stationary nor locally isotropic. Finally, at some spatial locations, there are very little data. We present a method called multiscale spatial density smoothing that successfully addresses these challenges. The method is based on recursive dyadic partition of the sample space, and therefore shares much in common with other multiscale methods, such as wavelets and Pólya-tree priors. We describe an efficient algorithm for finding a maximum a posteriori (MAP) estimate that leverages recent advances in convex optimization for nonsmooth functions. We apply multiscale spatial density smoothing to real data collected on the background gamma-ray spectra at locations across a large university campus. The method exhibits state-of-the-art performance for spatial smoothing in density estimation, and it leads to substantial improvements in power when used in conjunction with existing methods for detecting the kinds of radiological anomalies that may have important consequences for public health and safety.

Multiagent learning through neuroevolution

Neuroevolution is a promising approach for constructing intelligent agents in many complex tasks such as games, robotics, and decision making. It is also well suited for evolving team behavior for many multiagent tasks. However, new challenges and opportunities emerge in such tasks, including facilitating cooperation through reward sharing and communication, accelerating evolution through social learning, and measuring how good the resulting solutions are. This paper reviews recent progress in these three areas, and suggests avenues for future work.

Accelerating evolution via egalitarian social learning

Social learning is an extension to evolutionary algorithms that enables agents to learn from observations of others in the population. Historically, social learning algorithms have employed a student-teacher model where the behavior of one or more high-fitness agents is used to train a subset of the remaining agents in the population. This paper presents ESL, an egalitarian model of social learning in which agents are not labeled as teachers or students, instead allowing any individual receiving a sufficiently high reward to teach other agents to mimic its recent behavior. We validate our approach through a series of experiments in a robot foraging domain, including comparisons of egalitarian social learning with baseline neuroevolution and a variant of student-teacher social learning. In a complex foraging task, ESL converges to near-optimal strategies faster than either benchmark approach, outperforming both by more than an order of magnitude. The results indicate that egalitarian social learning is a promising new paradigm for social learning in intelligent agents.

DeXteR --- An Extensible Framework for Declarative Parameter Passing in Distributed Object Systems

In modern distributed object systems, reference parameters are passed to a remote method based on their runtime type. We argue that such type-based parameter passing is limiting with respect to expressiveness, readability, and maintainability, and that parameter passing semantics should be decoupled from parameter types. We present declarative parameter passing, an approach that fully decouples parameter passing semantics from parameter types in distributed object systems. In addition, we describe DeXteR, an extensible framework for transforming a type-based remote parameter passing model to a declaration-based model transparently. Our framework leverages aspect-oriented and generative programming techniques to enable adding new remote parameter passing semantics, without requiring detailed understanding of the underlying middleware implementation. Our approach is applicable to both application and library code and incurs negligible performance overhead. We validate the expressive power of our framework by adding several non-trivial remote parameter passing semantics (i.e., copy-restore, lazy, streaming) to Java RMI.

Trailblazer: a tool for automated annotation refactoring

Since annotations were added to the Java language, many enterprise frameworks have been transitioning to using annotated Plain Old Java Objects (POJOs) in their latest releases. Our automated refactoring tool, Trailblazer, alleviates the maintenance burden of such annotation refactoring tasks. The tool implements a novel approach that leverages a machine learning algorithm to infer semantics-preserving rules that are then used to automatically transform legacy Java classes. Using Trailblazer involves two phases. First, given an XML-based framework application, a programmer creates an annotation-based version of the application by hand, with Trailblazer recording the programmers actions. Trailblazer then uses inductive learning to infer generalized upgrade rules. In the second phase, other programmers can apply the inferred general transformation rules to upgrade any other application that uses the same framework. Thus, once one developer has trailblazed through the hurdles of manually upgrading for a given framework, other developers can automatically follow along the beaten path. In this demonstration, we will use transparent persistence as our example domain to show how Trailblazer can infer generalized rules and then automatically upgrade a legacy enterprise application that uses EJB 2.0 XML configuration files, to use EJB 3.0 annotations.