Naoyasu Ubayashi

A large-scale empirical study of just-in-time quality assurance

Defect prediction models are a well-known technique for identifying defect-prone files or packages such that practitioners can allocate their quality assurance efforts (e.g., testing and code reviews). However, once the critical files or packages have been identified, developers still need to spend considerable time drilling down to the functions or even code snippets that should be reviewed or tested. This makes the approach too time consuming and impractical for large software systems. Instead, we consider defect prediction models that focus on identifying defect-prone (“risky”) software changes instead of files or packages. We refer to this type of quality assurance activity as “Just-In-Time Quality Assurance,” because developers can review and test these risky changes while they are still fresh in their minds (i.e., at check-in time). To build a change risk model, we use a wide range of factors based on the characteristics of a software change, such as the number of added lines, and developer experience. A large-scale study of six open source and five commercial projects from multiple domains shows that our models can predict whether or not a change will lead to a defect with an average accuracy of 68 percent and an average recall of 64 percent. Furthermore, when considering the effort needed to review changes, we find that using only 20 percent of the effort it would take to inspect all changes, we can identify 35 percent of all defect-inducing changes. Our findings indicate that “Just-In-Time Quality Assurance” may provide an effort-reducing way to focus on the most risky changes and thus reduce the costs of developing high-quality software.

Association aspects

We propose a linguistic mechanism for AspectJ-like languages that concisely associates aspect instances to object groups. The mechanism, which supports association aspects, extends the per-object aspects in AspectJ by allowing an aspect instance to be associated to a group of objects, and by providing a new pointcut primitive to specify aspect instances as execution contexts of advice. With association aspects, we can straightforwardly implement crosscutting concerns that have stateful behavior related to a particular group of objects. The new pointcut primitive can more flexibly specify aspect instances when compared against previous implicit mechanisms. The comparison of execution times between the programs with association aspects and the ones with regular AspectJ aspects revealed that the association aspects exhibited almost equivalent for the medium-sized configurations.

Aspect-oriented programming with model checking

Aspect-oriented programming (AOP) is a programming paradigm such that crosscutting concerns including synchronization policies, resource sharing and performance optimizations over objects are modularized as aspects that are separated from objects. A compiler, called a weaver, weaves aspects and objects together into a program. In AOP, however, it is not easy to verify the correctness of a woven program because crucial behaviors are strongly influenced by aspect descriptions. In order to deal with such problem, this paper proposes an automatic verification approach using model checking that verifies whether the woven program contains unexpected behaviors such as deadlocks. The objectives of this paper are as follows: 1) to verify the correctness of AOP-based programs using model checking, 2) to provide AOP-based model checking frameworks.

An adaptive object model with dynamic role binding

To achieve the goal of realizing object adaptation to environments, a new role-based model Epsilon and a language EpsilonJ is proposed. In Epsilon, an environment is defined as a field of collaboration between roles and an object adapts to the environment assuming one of the roles. Objects can freely enter or leave environments and belong to multiple environments at a time so that dynamic adaptation or evolution of objects is realized. Environments and roles are the first class constructs at runtime as well as at model description time so that separation of concerns is not only materialized as a static structure but also observed as behaviors. Environments encapsulating collaboration are independent reuse components to be deployed separately from objects. In this paper, the Epsilon model and the language are explained with some examples. The effectiveness of the model is illustrated by a case study on the problem of integrated systems. Implementation of the language is also reported.

An empirical study of just-in-time defect prediction using cross-project models

Prior research suggests that predicting defect-inducing changes, i.e., Just-In-Time (JIT) defect prediction is a more practical alternative to traditional defect prediction techniques, providing immediate feedback while design decisions are still fresh in the minds of developers. Unfortunately, similar to traditional defect prediction models, JIT models require a large amount of training data, which is not available when projects are in initial development phases. To address this flaw in traditional defect prediction, prior work has proposed cross-project models, i.e., models learned from older projects with sufficient history. However, cross-project models have not yet been explored in the context of JIT prediction. Therefore, in this study, we empirically evaluate the performance of JIT cross-project models. Through a case study on 11 open source projects, we find that in a JIT cross-project context: (1) high performance within-project models rarely perform well; (2) models trained on projects that have similar correlations between predictor and dependent variables often perform well; and (3) ensemble learning techniques that leverage historical data from several other projects (e.g., voting experts) often perform well. Our findings empirically confirm that JIT cross-project models learned using other projects are a viable solution for projects with little historical data. However, JIT cross-project models perform best when the data used to learn them is carefully selected.

RoleEP: role based evolutionary programming for cooperative mobile agent applications

Using mobile agent systems, cooperative distributed applications that run over the Internet can be constructed flexibly. However, there are some problems: it is difficult to understand collaborations among the agents as a whole, and it is difficult to define the behaviors of agents because they are dynamically influenced by their external context. So, in general, constructions of cooperative distributed applications based on mobile agent systems are considered as very hard and difficult works. In this paper, the concept of RoleEP (Role-based Evolutionary Programming) is proposed in order to alleviate these problems. RoleEP provides a systematic evolutionary programming style. In RoleEP, a field where a group of agents collaborate with each other is regarded as an environment, and a function that an agent assumes in an environment is defined as a role. Descriptions only concerning the collaborations among agents can be abstracted by environments. An object becomes an agent by binding itself with a role that is defined in an environment, and it then acquires the functions needed for collaborating with other agents that exist in the same environment. Distributed applications based on mobile agent systems, which may change their functions dynamically in order to adapt themselves to their external context, can be constructed by synthesizing environments dynamically.

Studying just-in-time defect prediction using cross-project models

Unlike traditional defect prediction models that identify defect-prone modules, Just-In-Time (JIT) defect prediction models identify defect-inducing changes. As such, JIT defect models can provide earlier feedback for developers, while design decisions are still fresh in their minds. Unfortunately, similar to traditional defect models, JIT models require a large amount of training data, which is not available when projects are in initial development phases. To address this limitation in traditional defect prediction, prior work has proposed cross-project models, i.e., models learned from other projects with sufficient history. However, cross-project models have not yet been explored in the context of JIT prediction. Therefore, in this study, we empirically evaluate the performance of JIT models in a cross-project context. Through an empirical study on 11 open source projects, we find that while JIT models rarely perform well in a cross-project context, their performance tends to improve when using approaches that: (1) select models trained using other projects that are similar to the testing project, (2) combine the data of several other projects to produce a larger pool of training data, and (3) combine the models of several other projects to produce an ensemble model. Our findings empirically confirm that JIT models learned using other projects are a viable solution for projects with limited historical data. However, JIT models tend to perform best in a cross-project context when the data used to learn them are carefully selected.

Separation of Concerns in Mobile Agent Applications

Using mobile agent systems, cooperative distributed applications that run over the Internet can be constructed flexibly. However, there are some problems: it is difficult to understand collaborations among agents and travels of individual agents as a whole because mobility/collaboration functions tend to be intertwined in the code; it is difficult to define behaviors of agents explicitly because they are influenced by their external context dynamically. Many aspects of mobility/ collaboration strategies including traveling, coordination constraints, synchronization constraints and security-checking strategies should be considered when mobile agent applications are constructed.In this paper, the concept of RoleEP (Role Based Evolutionary Programming) is proposed in order to alleviate these problems. In RoleEP, a field where a group of agents roam around hosts and collaborate with each other is regarded as an environment and mobility/collaboration functions that an agent should assume in an environment are defined as roles. An object becomes an agent by binding itself to a role that is defined in an environment, and acquires mobility/collaboration functions dynamically. RoleEP provides a mechanism for separating concerns about mobility/collaboration into environments and a systematic evolutionary programming style. Distributed applications based on mobile agent systems, which may change their functions dynamically in order to adapt themselves to their external context, can be constructed by synthesizing environments dynamically.

Magnet or sticky? an OSS project-by-project typology

For Open Source Software (OSS) projects, retaining existing contributors and attracting new ones is a major concern. In this paper, we expand and adapt a pair of population migration metrics to analyze migration trends in a collection of open source projects. Namely, we study: (1) project stickiness, i.e., its tendency to retain existing contributors and (2) project magnetism, i.e., its tendency to attract new contributors. Using quadrant plots, we classify projects as attractive (highly magnetic and sticky), stagnant (highly sticky, weakly magnetic), fluctuating (highly magnetic, weakly sticky), or terminal (weakly magnetic and sticky). Through analysis of the MSR challenge dataset, we find that: (1) quadrant plots can effectively identify at-risk projects, (2) stickiness is often motivated by professional activity and (3) transitions among quadrants as a project ages often coincides with interesting events in the evolution history of a project.

Objects as Actors Assuming Roles in the Environment

To achieve the goal of realizing object adaptation to environments, a new role-based model Epsilonand a language EpsilonJare proposed. In Epsilon, an environment is defined as a field of collaboration between roles and an object adapts to the environment assuming one of the roles. Objects can freely enter or leave environments and belong to multiple environments at a time so that dynamic adaptation or evolution of objects is realized. Environments and roles are the first class constructs at runtime as well as at model description time so that separation of concerns is not only materialized as a static structure but also observable as behaviors. Environments encapsulating collaboration are independent reuse components to be deployed separately from objects. In this paper, the Epsilon model and the language are explained with some examples. The effectiveness of the model is illustrated by a case study on the problem of integrated systems. Implementation of the language is also reported.