Bonita Sharif

Mining software repositories for traceability links

An approach to recover/discover traceability links between software artifacts via the examination of a software systems version history is presented. A heuristic-based approach that uses sequential-pattern mining is applied to the commits in software repositories for uncovering highly frequent co-changing sets of artifacts (e.g., source code and documentation). If different types of files are committed together with high frequency then there is a high probability that they have a traceability link between them. The approach is evaluated on a number of versions of the open source system KDE. As a validation step, the discovered links are used to predict similar changes in the newer versions of the same system. The results show highly precision predictions of certain types of traceability links.

An Eye Tracking Study on camelCase and under_score Identifier Styles

An empirical study to determine if identifier-naming conventions (i.e., camelCase and under_score) affect code comprehension is presented. An eye tracker is used to capture quantitative data from human subjects during an experiment. The intent of this study is to replicate a previous study published at ICPC 2009 (Binkley et al.) that used a timed response test method to acquire data. The use of eye-tracking equipment gives additional insight and overcomes some limitations of traditional data gathering techniques. Similarities and differences between the two studies are discussed. One main difference is that subjects were trained mainly in the underscore style and were all programmers. While results indicate no difference in accuracy between the two styles, subjects recognize identifiers in the underscore style more quickly.

An eye-tracking study on the role of scan time in finding source code defects

An eye-tracking study is presented that investigates how individuals find defects in source code. This work partially replicates a previous eye-tracking study by Uwano et al. [2006]. In the Uwano study, eye movements are used to characterize the performance of individuals in reviewing source code. Their analysis showed that subjects who did not spend enough time initially scanning the code tend to take more time finding defects. The study here follows a similar setup with added eye-tracking measures and analyses on effectiveness and efficiency of finding defects with respect to eye gaze. The subject pool is larger and is comprised of a varied skill level. Results indicate that scanning significantly correlates with defect detection time as well as visual effort on relevant defect lines. Results of the study are compared and contrasted to the Uwano study.

The impact of identifier style on effort and comprehension

A family of studies investigating the impact of program identifier style on human comprehension is presented. Two popular identifier styles are examined, namely camel case and underscore. The underlying hypothesis is that identifier style affects the speed and accuracy of comprehending source code. To investigate this hypothesis, five studies were designed and conducted. The first study, which investigates how well humans read identifiers in the two different styles, focuses on low-level readability issues. The remaining four studies build on the first to focus on the semantic implications of identifier style. The studies involve 150 participants with varied demographics from two different universities. A range of experimental methods is used in the studies including timed testing, read aloud, and eye tracking. These methods produce a broad set of measurements and appropriate statistical methods, such as regression models and Generalized Linear Mixed Models (GLMMs), are applied to analyze the results. While unexpected, the results demonstrate that the tasks of reading and comprehending source code is fundamentally different from those of reading and comprehending natural language. Furthermore, as the task becomes similar to reading prose, the results become similar to work on reading natural language text. For more “source focused” tasks, experienced software developers appear to be less affected by identifier style; however, beginners benefit from the use of camel casing with respect to accuracy and effort.

An eye tracking study on the effects of layout in understanding the role of design patterns

The effect of layout in the comprehension of design pattern roles in UML class diagrams is assessed. This work replicates and extends a previous study using questionnaires but uses an eye tracker to gather additional data. The purpose of the replication is to gather more insight into the eye gaze behavior not evident from questionnaire-based methods. Similarities and differences between the studies are presented. Four design patterns are examined in two layout schemes in the context of three open source systems. Fifteen participants answered a series of eight design pattern role detection questions. Results show a significant improvement in role detection accuracy and visual effort with a certain layout for the Strategy and Observer patterns and a significant improvement in role detection time for all four patterns. Eye gaze data indicates classes participating in a design pattern act like visual beacons when they are in close physical proximity and follow the canonical layout, even though they violate some general graph aesthetics.

Eye movements in code reading: relaxing the linear order

Code reading is an important skill in programming. Inspired by the linearity that people exhibit while natural language text reading, we designed local and global gaze-based measures to characterize linearity (left-to-right and top-to-bottom) in reading source code. Unlike natural language text, source code is executable and requires a specific reading approach. To validate these measures, we compared the eye movements of novice and expert programmers who were asked to read and comprehend short snippets of natural language text and Java programs. Our results show that novices read source code less linearly than natural language text. Moreover, experts read code less linearly than novices. These findings indicate that there are specific differences between reading natural language and source code, and suggest that non-linear reading skills increase with expertise. We discuss the implications for practitioners and educators.

Eye tracking in computing education

The methodology of eye tracking has been gradually making its way into various fields of science, assisted by the diminishing cost of the associated technology. In an international collaboration to open up the prospect of eye movement research for programming educators, we present a case study on program comprehension and preliminary analyses together with some useful tools. The main contributions of this paper are (1) an introduction to eye tracking to study programmers; (2) an approach that can help elucidate how novices learn to read and understand programs and to identify improvements to teaching and tools; (3) a consideration of data analysis methods and challenges, along with tools to address them; and (4) some larger computing education questions that can be addressed (or revisited) in the context of eye tracking.

On the use of eye tracking in software traceability

The paper advocates for the induction of eye tracking technology in software traceability and takes a position that the use of eye tracking metrics can contribute to several software traceability tasks. The authors posit that the role of eye tracking is not simply restricted to an instrument for empirical studies, but also could extend to providing a foundation of a new software traceability methodology. Several scenarios where eye-tracking metrics could be meaningful are presented. The specific research directions include conducting empirical studies with eye-tracking metrics and replicating previously reported empirical studies, eye-tracking enabled traceability link recovery and management methodology, and visualization support.

Improving the accuracy of duplicate bug report detection using textual similarity measures

The paper describes an improved method for automatic duplicate bug report detection based on new textual similarity features and binary classification. Using a set of new textual features, inspired from recent text similarity research, we train several binary classification models. A case study was conducted on three open source systems: Eclipse, Open Office, and Mozilla to determine the effectiveness of the improved method. A comparison is also made with current state-of-the-art approaches highlighting similarities and differences. Results indicate that the accuracy of the proposed method is better than previously reported research with respect to all three systems.

The effect of layout on the comprehension of UML class diagrams: A controlled experiment

The results of a controlled experiment assessing the effects of different layout strategies on the comprehension of UML class diagrams of two software systems is presented. Six different categories of software comprehension tasks, with varying degrees of difficulty, are used to assess the layouts. Each task consists of several questions aimed at measuring the comprehensibility of a layout. The study involved 45 participants of varied experience in software design and programming ability. A report on the quantitative analysis of accuracy, speed, confidence level and preference of solving the tasks is given. Results indicate that clustered layouts demonstrate significant improvement in subject accuracy and speed in solving the problems in a majority of tasks.