Siyuan Jiang

DUA-forensics: a fine-grained dependence analysis and instrumentation framework based on Soot

We describe DUA-Forensics, our open-source Java-bytecode program analysis and instrumentation system built on top of Soot. DUA-Forensics has been in development for more than six years and has supported multiple research projects on efficient monitoring, test-suite augmentation, fault localization, symbolic execution, and change-impact analysis. Three core features of Soot have proven essential: the Java bytecode processor, the Jimple intermediate representation, and the API to access and manipulate Jimple programs. On top of these foundations, DUA-Forensics offers a number of features of potential interest to the Java-analysis community, including (1) a layer that facilitates the instrumentation of Jimple code, (2) a library modeling system for efficient points-to, data-flow, and symbolic analysis, and (3) a fine-grained dependence analysis component. These features have made our own research more productive, reliable, and effective.

Quantitative program slicing: separating statements by relevance

Program slicing is a popular but imprecise technique for identifying which parts of a program affect or are affected by a particular value. A major reason for this imprecision is that slicing reports all program statements possibly affected by a value, regardless of how relevant to that value they really are. In this paper, we introduce quantitative slicing (q-slicing), a novel approach that quantifies the relevance of each statement in a slice. Q-slicing helps users and tools focus their attention first on the parts of slices that matter the most. We present two methods for quantifying slices and we show the promise of q-slicing for a particular application: predicting the impacts of changes.

SENSA: Sensitivity Analysis for Quantitative Change-Impact Prediction

Sensitivity analysis determines how a system responds to stimuli variations, which can benefit important software-engineering tasks such as change-impact analysis. We present SENSA, a novel dynamic-analysis technique and tool that combines sensitivity analysis and execution differencing to estimate the dependencies among statements that occur in practice. In addition to identifying dependencies, SENSA quantifies them to estimate how much or how likely a statement depends on another. Quantifying dependencies helps developers prioritize and focus their inspection of code relationships. To assess the benefits of quantifying dependencies with SENSA, we applied it to various statements across Java subjects to find and prioritize the potential impacts of changing those statements. We found that SENSA predicts the actual impacts of changes to those statements more accurately than static and dynamic forward slicing. Our SENSA prototype tool is freely available for download.

Automated Coverage-Driven Test Data Generation Using Dynamic Symbolic Execution

Recently code transformations or tailored fitness functions are adopted to achieve coverage (structural or logical criterion) driven testing to ensure software reliability. However, some internal threats like negative impacts on underlying search strategies or local maximum exist. So we propose a dynamic symbolic execution (DSE) based framework combined with a path filtering algorithm and a new heuristic path search strategy, i.e., predictive path search, to achieve faster coverage-driven testing with lower testing cost. The empirical experiments (three open source projects and two industrial projects) show that our approach is effective and efficient. For the open source projects w.r.t branch coverage, our approach in average reduces 25.5% generated test cases and 36.3% solved constraints than the traditional DSE-based approach without path filtering. And the presented heuristic strategy, on the same testing budget, improves the branch coverage by 26.4% and 35.4% than some novel search strategies adopted in KLEE and CREST.

Detecting user story information in developer-client conversations to generate extractive summaries

User stories are descriptions of functionality that a software user needs. They play an important role in determining which software requirements and bug fixes should be handled and in what order. Developers elicit user stories through meetings with customers. But user story elicitation is complex, and involves many passes to accommodate shifting and unclear customer needs. The result is that developers must take detailed notes during meetings or risk missing important information. Ideally, developers would be freed of the need to take notes themselves, and instead speak naturally with their customers. This paper is a step towards that ideal. We present a technique for automatically extracting information relevant to user stories from recorded conversations between customers and developers. We perform a qualitative study to demonstrate that user story information exists in these conversations in a sufficient quantity to extract automatically. From this, we found that roughly 10.2% of these conversations contained user story information. Then, we test our technique in a quantitative study to determine the degree to which our technique can extract user story information. In our experiment, our process obtained about 70.8% precision and 18.3% recall on the information.

Do Programmers do Change Impact Analysis in Debugging

Abstract“Change Impact Analysis” is the process of determining the consequences of a modification to software. In theory, change impact analysis should be done during software maintenance, to make sure changes do not introduce new bugs. Many approaches and techniques are proposed to help programmers do change impact analysis automatically. However, it is still an open question whether and how programmers do change impact analysis. In this paper, we conducted two studies, one in-depth study and one breadth study. For the in-depth study, we recorded videos of nine professional programmers repairing two bugs for two hours. For the breadth study, we surveyed 35 professional programmers using an online system. We found that the programmers in our studies did static change impact analysis before they made changes by using IDE navigational functionalities, and they did dynamic change impact analysis after they made changes by running the programs. We also found that they did not use any change impact analysis tools.

On the Accuracy of Forward Dynamic Slicing and Its Effects on Software Maintenance

Dynamic slicing is a practical and popular analysis technique used in various software-engineering tasks. Dynamic slicing is known to be incomplete because it analyzes only a subset of all possible executions of a program. However, it is less known that its results may inaccurately represent the dependencies that occur in those executions. Some researchers have identified this problem and developed extensions such as relevant slicing, which incorporates static information. Yet, dynamic slicing continues to be widely used, even though the extent of its inaccuracy is not well understood, which can affect the benefits of this analysis. In this paper, we present an approach to assess the accuracy of forward dynamic slices, which are used in software maintenance and evolution tasks. Because finding all actual dependencies is an undecidable problem, our approach instead computes bounds of the precision and recall of forward dynamic slices. Our approach uses sensitivity analysis and execution differencing to find a subset of all program statements that truly depend at runtime on another statement. Using this approach, we studied the accuracy of many forward dynamic slices from a variety of Java applications. Our results show that forward dynamic slicing can have low recall -- for dependencies in the analyzed executions -- and some potential imprecision. We also conducted a case study that shows how this inaccuracy affects a software maintenance task. To the best of our knowledge, ours is the first work that quantifies the intrinsic limitations of dynamic slicing.

Prioritizing Change-Impact Analysis via Semantic Program-Dependence Quantification

Software is constantly changing. To ensure the quality of this process, when preparing to change a program, developers must first identify the main consequences and risks of modifying the program locations they intend to change. This activity is called change-impact analysis. However, existing impact analysis suffers from two major problems: coarse granularity and large size of the resulting impact sets. Finer-grained analyses such as slicing give more detailed impact sets which, however, are also even larger in size. While various impact-set reduction approaches have been proposed at different levels of granularity, the challenge persists as very-large impact sets are still produced, impeding the adoption of impact analysis due to the great costs of inspecting those impact sets. To address these challenges, we present a novel dynamic-analysis technique called SensA which combines sensitivity analysis and execution differencing. SensA not only provides fine-grained (statement-level) impact sets but also prioritizes potential impacts via semantic-dependence quantification for program slices. We evaluated the benefits of impact prioritization using SensA with respect to static and dynamic forward slicing via an extensive empirical study of open-source Java applications and three case studies. Our results show that SensA can offer much better cost-effectiveness than slicing in assisting developers with impact inspection and fault cause-effect understanding.

Change-Effects Analysis for Evolving Software

Abstract Software constantly changes during its life cycle. This phenomenon is particularly prominent in modern software, whose complexity keeps growing and changes rapidly in response to market pressures and user demands. At the same time, developers must assure the quality of this software in a timely manner. Therefore, it is of critical importance to provide developers with effective tools and techniques to analyze, test, and validate their software as it evolves. While techniques for supporting software evolution abound, a conceptual foundation for understanding, analyzing, comparing, and developing new techniques is also necessary for the continuous growth of this field. A key challenge for many of these techniques is to accurately model and compute the effects of changes on the behavior of software systems. Such a model helps understand, compare, and further advance important activities such as change-impact analysis, regression testing, test-suite augmentation, and program merging. To address this challenge, in this chapter, we describe our progress and those of other researchers in developing and studying a foundational approach called change-effects analysis . This kind of analysis computes, in an ideal setting (i.e., given unlimited resources), all the differences that a change in the code of a program can cause on every element (e.g., statement) of that program. These differences include not only which program elements are affected by a change, but also how exactly their behavior (i.e., execution frequency and states) is affected. Naturally, computing all possible effects of changes is an undecidable problem. However, such a model for computing change effects can be the basis for practical approaches that approximate those effects. In this chapter, we describe various such approximations for change-effects analysis and their application to two key tasks in software evolution: change-impact analysis and change testing.

A Parallel Approach to Concolic Testing with Low-cost Synchronization

This paper presents a practical approach to parallelize the test data generation algorithm by which computing resources can be fully used. The test data generation approach that we are using is based on the dynamic symbolic execution (concolic testing). The basic idea of parallelizing the algorithm is to distribute analysis processes of different paths to different computing units. Although a centralized scheduler with several sub processes can directly achieve the goal of parallelism, it may cause global idle time when parallel processes frequently end at same time. In our approach, a runtime deterministic scheduler is introduced to reduce the potential global idle time. Our experiments show some notable results when using a proper scheduling function. Compared with the sequential concolic testing, our approach can save nearly 70% computing time in some cases on a system with eight CPU cores from our experiments.