Oleksii Kononenko

The influence of non-technical factors on code review

When submitting a patch, the primary concerns of individual developers are “How can I maximize the chances of my patch being approved, and minimize the time it takes for this to happen?” In principle, code review is a transparent process that aims to assess qualities of the patch by their technical merits and in a timely manner; however, in practice the execution of this process can be affected by a variety of factors, some of which are external to the technical content of the patch itself. In this paper, we describe an empirical study of the code review process for WebKit, a large, open source project; we replicate the impact of previously studied factors - such as patch size, priority, and component and extend these studies by investigating organizational (the company) and personal dimensions (reviewer load and activity, patch writer experience) on code review response time and outcome. Our approach uses a reverse engineered model of the patch submission process and extracts key information from the issue tracking and code review systems. Our findings suggest that these nontechnical factors can significantly impact code review outcomes.

The MSR Cookbook: Mining a decade of research

The Mining Software Repositories (MSR) research community has grown significantly since the first MSR workshop was held in 2004. As the community continues to broaden its scope and deepens its expertise, it is worthwhile to reflect on the best practices that our community has developed over the past decade of research. We identify these best practices by surveying past MSR conferences and workshops. To that end, we review all 117 full papers published in the MSR proceedings between 2004 and 2012. We extract 268 comments from these papers, and categorize them using a grounded theory methodology. From this evaluation, four high-level themes were identified: data acquisition and preparation, synthesis, analysis, and sharing/replication. Within each theme we identify several common recommendations, and also examine how these recommendations have evolved over the past decade. In an effort to make this survey a living artifact, we also provide a public forum that contains the extracted recommendations in the hopes that the MSR community can engage in a continuing discussion on our evolving best practices.

The Secret Life of Patches: A Firefox Case Study

The goal of the code review process is to assess the quality of source code modifications (submitted as patches) before they are committed to a projects version control repository. This process is particularly important in open source projects to ensure the quality of contributions submitted by the community, however, the review process can promote or discourage these contributions. In this paper, we study the patch lifecycle of the Mozilla Fire fox project. The model of a patch lifecycle was extracted from both the qualitative evidence of the individual processes (interviews and discussions with developers), and the quantitative assessment of the Mozilla process and practice. We contrast the lifecycle of a patch in pre- and post-rapid release development. A quantitative comparison showed that while the patch lifecycle remains mostly unchanged after switching to rapid release, the patches submitted by casual contributors are disproportionately more likely to be abandoned compared to core contributors. This suggests that patches from casual developers should receive extra care to both ensure quality and encourage future community contributions.

Automatically locating relevant programming help online

While maintaining software systems, developers often encounter compilation errors and runtime exceptions that they do not know how to solve. Solutions to these errors can often be found through discussions with other developers on the Internet. Unfortunately, many of these online discussions do not contain relevant answers. We have developed an approach to automatically query and analyze online discussions to locate relevant solutions to programming problems. Our tool, called Dora, is integrated into Visual Studio and allows developers to query and evaluate solutions within their development environment, enabling them to reduce context switching between their development tasks and their search sessions. We have performed a semi-controlled experiment to validate the utility of our search approach with 18 tasks, finding that our approach provides 55% more relevant results than traditional web searching approaches.

Investigating technical and non-technical factors influencing modern code review

When submitting patches for code review, individual developers are primarily interested in maximizing the chances of their patch being accepted in the least time possible. In principle, code review is a transparent process in which reviewers aim to assess the qualities of the patch on its technical merits in a timely manner; however, in practice the execution of this process can be affected by a variety of factors, some of which are external to the technical content of the patch itself. In this paper, we describe empirical studies of the code review processes for large, open source projects such as WebKit and Google Blink. We first consider factors that have been examined in previous studies — patch size, priority, and component — and then extend our enquiries to explore the effects of organization (which company is involved) and developer profile (review load and activity, patch writer experience) on code review response time and eventual outcome. Our approach uses a reverse engineered model of the patch submission process, and extracts key information from the issue-tracking and code review systems. Our findings suggest that these non-technical factors can significantly impact code review outcomes.

Code review quality: how developers see it

In a large, long-lived project, an effective code review process is key to ensuring the long-term quality of the code base. In this work, we study code review practices of a large, open source project, and we investigate how the developers themselves perceive code review quality. We present a qualitative study that summarizes the results from a survey of 88 Mozilla core developers. The results provide developer insights into how they define review quality, what factors contribute to how they evaluate submitted code, and what challenges they face when performing review tasks. We found that the review quality is primarily associated with the thoroughness of the feedback, the reviewers familiarity with the code, and the perceived quality of the code itself. Also, we found that while different factors are perceived to contribute to the review quality, reviewers often find it difficult to keep their technical skills up-to-date, manage personal priorities, and mitigate context switching.

Compiling Clones: What Happens?

Most clone detection techniques have focused on the analysis of source code, however, sometimes stakeholders have access only to compiled code. To address this, some approaches have been developed for finding similarities at the binary level. However, the precise relationships between source-level and binary-level similarities remains unclear: While a compiler will preserve the semantics of the source code in the transformation to an executable, the resulting binary may differ significantly in structure, including the addition and deletion of entities in the source model. Also, compilation sometimes acts as a kind of normalization, transforming syntactically different but semantically similar structures into the same binary-level representation. In this paper, we describe a preliminary study into the effects of the javac Java compiler on the results of clone detection. We use CCFinderX -- which can perform clone detection on sequences of arbitrary tokens -- to find clones in both the source code and the corresponding byte code of four large Java systems. The study shows that source code and byte code clone detection can produce significantly different results, especially for large programs. We report on a few typical examples of differences, and analyze how they are introduced by the compiler. Finally, we discuss the greater significance of this work, and sketch plans for expanded study.

Extracting artifact lifecycle models from metadata history

Software developers and managers make decisions based on the understanding they have of their software systems. This understanding is both built up experientially and through investigating various software development artifacts. While artifacts can be investigated individually, being able to summarize characteristics about a set of development artifacts can be useful. In this paper we propose lifecycle models as an effective way to gain an understanding of certain development artifacts. Lifecycle models capture the dynamic nature of how various development artifacts change over time in a graphical form that can be easily understood and communicated. Lifecycle models enables reasoning of the underlying processes and dynamics of the artifacts being analyzed. In this paper we describe how lifecycle models can be generated and demonstrate how they can be applied to the code review process of a development project.

Synthesizing Knowledge from Software Development Artifacts

Abstract When software practitioners make day-to-day design decisions about their projects, they are guided by not only their intuition and experience, but also by the variety of software artifacts that are available to them. This chapter describes how lifecycle models can be used to build a useful and intuitive model of these development artifacts. Lifecycle models capture the dynamic nature of how such artifacts change over time in a graphical form that can be easily understood and communicated. We show how lifecycle models can be generated, and we present two industrial case studies where we applied lifecycle models to assess a project’s code review process.