Developer Prioritization in Bug Repositories
DDeveloper Prioritization in Bug Repositories
Jifeng Xuan [email protected]
He Jiang [email protected]
Zhilei Ren [email protected]
Weiqin Zou [email protected]
School of Software, Dalian University of Technology Dalian, China
Abstract —Developers build all the software artifacts in de-velopment. Existing work has studied the social behavior in software repositories. In one of the most important software repositories, a bug repository, developers create and update bug reports to support software development and maintenance. However, no prior work has considered the priorities of devel-opers in bug repositories. In this paper, we address the prob-lem of the developer prioritization, which aims to rank the contributions of developers. We mainly explore two aspects, namely modeling the developer prioritization in a bug reposi-tory and assisting predictive tasks with our model. First, we model how to assign the priorities of developers based on a social network technique. Three problems are investigated, including the developer rankings in products, the evolution over time, and the tolerance of noisy comments. Second, we consider leveraging the developer prioritization to improve three predicted tasks in bug repositories, i.e., bug triage, sever-ity identification, and reopened bug prediction. We empirically investigate the performance of our model and its applications in bug repositories of Eclipse and Mozilla. The results indicate that the developer prioritization can provide the knowledge of developer priorities to assist software tasks, especially the task of bug triage.
Keywords- developer prioritization; software evolution; bug triage; severity identification; reopened bug prediction I. I NTRODUCTION
A bug repository is a vital database in modern software development. Many software projects create and maintain bug repositories for storing and updating the information of problems or suggestions about projects [1]. The widely available bug repositories have provided an important plat-form for investigating the quality of software [16]. With the growth in scale, developers in large projects must handle a large number of bugs in bug repositories. For example, from Oct. 2001 to Dec. 2010, the bug repository of an open source project, Eclipse [12], has recorded 333371 bugs, which are totally commented for 1544996 times by 34917 contributors around the world. In
Peopleware , Demarco & Lister [11] have proposed “
The major problems of our work are not so much techno-logical as sociological in nature ”. All the software artifacts in software repositories are created, updated, and studied by people. The social behavior of people has a significant im-pact on software development. Existing work has examined the social networks for some kinds of software repositories. For mailing list repositories, Bird et al. mine communication networks and discover the community structure from email archives [8]; Wolf et al. predict software build failures using social networks measures on the developer communication [34]. For change log repositories, Meneely et al. [25] and Pinzger et al. [27] build developer networks to predict soft-ware failures. For bug repositories, Hong et al. have divided the developer network into several communities, which iden-tify the sub-groups of developer communication [17]. How-ever, no prior work has considered the priorities of develop-ers in bug repositories and its applications. In this paper, we model the developer prioritization using a socio-technical approach to improve three predicting tasks centering on bug repositories. In contrast to dividing the de-veloper network into communities in [17], we generate the developer prioritization by ranking all the participant devel-opers of bug repositories. Based on our approach, we further study four Research Questions (RQs) with the experiments on two typical open source projects, namely Eclipse and Mozilla. We analyze the characteristics of developer prioriti-zation to address the first three RQs, which study the devel-oper priorities in products, the evolution over time, and the tolerance of noises, respectively; on the other hand, we ad-dress the last RQ by leveraging the developer prioritization to improve three typical tasks of bug repositories, including bug triage, severity identification, and reopened bug predic-tion. The experiments show that the developer prioritization is helpful to improve the predicting tasks in bug repositories. Especially, for bug triage, the average accuracy is improved up to 13% by combining the developer prioritization. The primary contributions of this paper are as follows: 1. We identify the developer prioritization of bug reposi-tories based on a socio-technical approach. In our work, each developer is mapped to a probability to indicate the priority in software development. To our knowledge, this is the first work for ranking developers with social networks in bug repositories. 2. We present detailed analysis of our developer prioriti-zation. In the analysis, we examine the characteristics of the developer prioritization in bug repositories, including the developer priorities in products, the evolution, and the toler-ance of noises. 3. We explore how to improve the tasks in bug reposito-ries. We present the results of three typical tasks, i.e., im-proving bug triage by mixing the developer priorities, identi-fying bug severity by adding new features, and predicting reopened bugs by changing metrics. To our knowledge, this is the first work to evaluate the results of socio-techniques on bug-related tasks. The remainder of this paper is organized as follows. Sec-tion II states the background. Section III shows the approach o recognize the developer prioritization. In Section IV, we study the characteristics of developer prioritization on bug repositories. In Section V, we describe how to improve soft-ware tasks by leveraging the developer prioritization. Section VI gives the threats to validity and Section VII shows the related work. Section VIII briefly summarizes this paper and presents the future work. II. B ACKGROUND
Bug repositories (also known as bug tracking systems ) are deployed in software projects for the storage and man-agement of bugs. A bug in bug repositories is recorded as a bug report , which is filled with the information of a software problem. Based on the bug reports, developers can collect and reproduce bugs for bug fixing [40]. In empirical software engineering, many software tasks are investigated on bug repositories, e.g., assigning bugs to correct developers to reduce the time of bug fixing [1], [19], [32], summarizing a long bug report into a short abstract [29], detecting duplicate bugs to avoid repetitive operations [28], [35], [31], and char-acterizing factors of bug fixing [15]. In bug repositories, a bug report belongs to a component of a product. A product consists of multiple components and some products may share a common component. For exam-ple, Products firefox and core in Mozilla [22] use a common component, file handling . During the process of bug fixing, developers label a bug report with various statuses to denote a bug report as new, assigned, resolved, reopened, etc [18]. Once a bug report is created, any developer, who is interest-ed in this bug, can make comments to communicate with relevant developers. In existing work on bug repositories, developers are always treated equally. However, the priority of a developer plays an important role in the tasks. For ex-ample, an active developer may make more contributions on bug fixing than an inactive one; an experienced tester may find bugs with higher severities than a common end user. In this paper, we rank all the developers of bug repositories to assist the tasks around bug repositories. We denote the pro-cess of generating the developer priorities as developer pri-oritization . In this paper, “developers” refers to the people who contribute to a bug repository. We follow existing work [17], [19] to use the term developers in a broad sense, including reporters, programmers, testers and active end users. We denote a developer with the user name in the email instead of the real name.
We take two bug reports (bug IDs 261871 and 264696) in Eclipse as examples to illustrate the developer prioritiza-tion. Both of these bugs are fixed bugs in Product
Plug-in Development Environment (PDE) . In Table I, we list the details of these two bugs.
Each bug report is reported by one developer and commented by developers for 7 times. We find that these two bugs share two common developers, caniszczyk and cwindatt . We illustrate the process of com-menting among the developers in Fig. 1. An arrow with a value in Fig. 1 is called a link . A question for us is how to prioritize the five developers for these two bug reports. This question is not easy to answer. Intuitively, both caniszczyk and cwindatt contribute to two bug reports while the other developers contribute to only one. On the other hand, ankur_sharma , cwindatt , and bcabe are very active since there are 3 links from each of them and 3 links to each of them (also a self-link for cwindatt ). In this paper, we aim to model the developer prioritization. Fur-thermore, since developer factors are important for the met-rics of software quality [40], another question is how to im-prove the software quality with the developer prioritization. In our work, we want to extract knowledge from the devel-oper priorities to assist software development. III. I DENTIFYING THE D EVELOPER P RIORITIZATION
In this section, we model the developer prioritization by extending a social network technique. We present the framework of developer prioritization and propose four re-search questions for further studies. A. Framework of Developer Prioritization
Motivated by Fig. 1, we define the developer prioritiza-tion in our work.
Developer prioritization is a process to assign a priority to each developer in a bug repository and to rank all the con-tributions of developers to assist software tasks. We extend a recent socio-technical approach proposed by Lü et al. [21] to identify the developer prioritization. In their work, a leadership network is proposed to investigate the social analysis between leaders and fans on an online book-marking website. In our developer prioritization, we transfer their work to developers in a bug repository. We contribute in two extensions. First, we adapt the original binary weights of links to integer weights based on the number of comments.
TABLE Ⅰ T WO B UG R EPORTS IN E CLIPSE
Commenter order
Bug ID 261871 Bug ID 264696
Developer Date & time Developer Date & time
Reporter olivier_thomann 09-01-21 12:39 cwindatt 09-02-12 10:02 Commenter 1 cwindatt 09-01-21 12:54 cwindatt 09-03-12 16:25 Commenter 2 bcabe 09-01-21 12:56 ankur_sharma 09-04-03 16:24 Commenter 3 olivier_thomann 09-01-21 13:40 cwindatt 09-04-06 10:31 Commenter 4 bcabe 09-01-21 14:08 ankur_sharma 09-04-14 17:01 Commenter 5 olivier_thomann 09-01-21 14:18 caniszczyk 09-04-14 19:53 Commenter 6 bcabe 09-01-21 15:43 ankur_sharma 09-04-15 06:49 Commenter 7 caniszczyk 09-01-23 10:58 cwindatt 09-04-15 12:49
Figure 1.
An illustration of the communication among developers. A link denotes a comment and the value of a link denotes the times of comments, e.g., the link with a value 1 from bcabe to caniszczyk means that caniszczyk has made one comment on bcabe (based on Commenters 6 and 7 in bug ID 261871). caniszczykcwindatt ankur_sharmabcabeolivier_thomann econd, we propose a topic-based prioritization by specify-ing a product or a component. We present the brief framework of the developer prioriti-zation in Algorithm 1. Given developers in a bug reposito-ry, the goal of developer prioritization is to generate a score for each developer and rank all the developers based on these scores. A weight denotes the number of all the comments in a link from a developer to . If no link ex-ists between and , . Specially, we remove all the self-links since we omit the influence of comments from a developer to himself/herself (e.g., the self-link of cwindatt in Fig. 1). To build a connected graph based on all the links, a virtual developer is added to connect all the developers. Then, we add a bi-directional link between each original developer and . The weights of this link, and , are set to 1 for each ( ). For each developer, let ( ) denote the out-degree of . Note that both out-degrees and in-degrees are informative to study the so-cial networks in bug repositories [17]. In our work, we use out-degrees to reflect the influences of both reporters and commenters. We consider that the developer prioritization is generated based on the changes of time series. We denote as the score of developer ( ) at time . Thus, we calcu-late this score, (1) The initial score for and . Lü et al. have proved that the above model can converge after finite time [21]. Given the convergence time , we generate the final score of ( ), (2) where is a parameter for normalization and . Thus, each developer in the bug repository is assigned with a score ( ). We rank all the participant developers by their scores in descending order. In other words, a developer in a top rank owns a higher priority than a developer in a bottom rank. We apply the developer prioritization to the example in Fig. 1. The scores of the five developers are , , , , and , respectively; the developer ankur_sharma has the highest priority among these five developers. The scores of these five developers illustrate that Algo-rithm 1 can quantify the priorities of developers. In contrast to the same value by measuring out-degrees (e.g., the same out-degree of ankur_sharma , cwindatt , and bcabe ), Algo-rithm 1 distinguishes the developers with the same out-degree. Based on Algorithm 1, the developer prioritization can be adapted to a topic-based model. In a bug repository, we usu-ally focus on a specified product or component of the project. A straightforward choice is to denote a topic with a product or a component. For example, given a product, we can gen-erate the developer priorities based on the bug comments related to this product. Product-based or component-based developer prioritization is helpful to study the priorities of developers in a specified part of the project. B. Research Questions
We propose four Research Questions (RQs) to investi-gate the developer prioritization. These four RQs are divided into two categories, namely the characteristic analysis and the applications. We answer these two categories of RQs in Section IV (RQ1-RQ3) and Section V (RQ4), respectively.
RQ1 . Does the developer prioritization for the whole project differ from the one for a product?
RQ2 . How does the developer prioritization evolve over time?
RQ3 . Is the developer prioritization tolerant to noisy comments?
To analyze the characteristics of the developer prioritiza-tion, in RQ1, we study the differences of the developer prior-itization between the products and the whole project; in RQ2, we study the evolution of the developer prioritization over time; in RQ3, we examine the tolerance of noises for the developer prioritization.
RQ4 . Can we use the developer prioritization to assist the existing tasks in bug repositories?
An important problem is to explore the applications of the developer prioritization. In RQ4, we investigate how to incorporate the developer prioritization to improve typical tasks in bug repositories. IV. A NALYZING THE D EVELOPER P RIORITIZATION
To explore the answers to the above four RQs, we con-duct experiments on bug repositories of two open source projects, Eclipse and Mozilla. In this section, we present the details of the data collection and investigate the answers to RQ1 - RQ3. A. Data Collection
We analyze the characteristics of the developer prioritiza-tion based on bug repositories of Eclipse and Mozilla. These two projects have attracted wide interests since both of them are large scale and open source projects. In our work, we
Algorithm 1 . Framework of Developer Prioritization
Input: developer ( ), links among Output: final score for each developer add a virtual developer and add bi-directional links with ; set initial scores ( ) and ; for to do // is the time for convergence calculate the score of each ( ) at time with (1); calculate the final score of each ( ) with (2). TABLE Ⅱ S CALES OF D ATA S ETS
Projects
Period Eclipse
Mozilla ollect all the bug reports before 2011, i.e., bugs 1-333371 in Eclipse and bugs 1-662320 in Mozilla. The details of our data sets are listed in Table II. Note that the number of bug reports does not equal to the range of bug IDs since some bug reports are removed in development (e.g., bug 5315 in Eclipse) or not anonymously accessible (e.g., bug 400020 in Mozilla). For each bug report, we extract the bug ID, the reporter, the fixer, the summary, the description, the creating time, and the comments. For each comment on a bug report, we extract the commenter and the commenting time. Since the fixer of a bug report is not always correctly labeled, the heu-ristics in [1] are used to recognize the correct fixers in bug repositories. In this section, we mainly focus on the com-ments of bug reports while in Section V, we will further study the texts of bug reports, e.g., the summary and the de-scription. B. Answer to RQ1, Developer Prioritization in Products
Most open source projects consist of multiple products, each of which can be viewed as a sub-project for a set of individual requirements. In practice, a developer can partici-pate in multiple products since the experience from one product may guide the development of another one. In this sub-section, we examine the changes of developer prioritiza-tion between the whole project and its products. Before studying the developer prioritization for products, we first illustrate the developer rankings in the whole pro-jects in Fig. 2. Three indicators of developers are used, namely the rank in the developer prioritization, the out-degree in a bug repository, and the number of fixed bugs. We choose the out-degree as an indicator since the developer prioritization is constructed based on the out-degrees of de- velopers (in Algorithm 1). Note that since not all the bugs are fixed in bug repositories, we only count the number of fixed bugs based on bug reports with the resolution “fixed” to simplify the statistics. In Fig. 2, a developer with a large out-degree leads to a high priority in developer rankings. Moreover, most of de-velopers with high priorities have fixed a large amount of bugs, e.g., most of the largest circles for developers (who have fixed over 1500 bugs) lie in top 40 ranks. We can ob-serve that the curve of Eclipse is not as stable as that of Mozilla. In Eclipse, most of the out-degrees are over 2000. In both Eclipse and Mozilla, there are some developers who are dominant in both the priority and the number of fixed bugs, i.e., top 2 developers in Eclipse and top 4 developers in Mozilla. These dominant developers may be the experienced experts in software development [2]. Note that some devel-opers, who have only fixed a small number of bugs, also have high priorities, e.g., a developer in Mozilla is ranked in top 30 and has fixed less than 500 bugs. This fact is caused by the different duties of developers, e.g., an active develop-er may be not a fixer but a tester. To observe the differences of developer prioritization be-tween the whole project and its products, we show the ranks in 10 products for 5 developers in Fig. 3, who are ranked as the top 5 in the whole project (i.e., top 5 developers in Fig. 2). The curves for most developers have the similar trend, e.g., in Mozilla, the 5 developers have high priorities in Product core and have low priorities in Product calendar . An excep-tion is the rank for Product wtp source editing in Eclipse, which widely distributes between 1 and 1000. Moreover, a top developer in the whole project may contribute little to a product or not participate in a product, e.g., in Eclipse, the developer john_arthorne has ranked around 900 in Product birt and the developer darin.eclipse has not contributed to birt . (a) Eclipse (b) Mozilla Figure 2. Out-degrees for top 100 developers. The diameter of a circle denotes the number of bug reports for a developer. For example, the smallest circles denote developers who have fixed less than 500 bugs. Note that the vertical axis is on a log scale. (a) Eclipse (b) Mozilla
Figure 3.
Ranks in 10 active products for top 5 developers in the whole project. For each project, the selected products are 10 products, which are contributed by most developers. In horizontal axis, the number following with a product name denotes the number of participant developers. Note that values in the vertical axis are in reverse order. O u t- d e g r ee o f a d eve l op e r Rank O u t- d e g r ee o f a d eve l op e r Rank l a tf o r m ( ) j d t ( ) c d t ( ) b i r t ( ) equ i no x ( ) pde ( ) c o mm un i t y ( ) w t p s ou r c e ed i t i ng ( ) m y l y n ( ) e m f ( ) R an k Product namedaniel_megert darin.eclipsejohn_arthorne eclipse david_williams f i r e f o x ( ) c o r e ( ) s ea m on k e y ( ) t hunde r b i r d ( ) m a il ne w s c o r e ( ) t oo l k i t ( ) t e c h e v ange li s m ( ) c o r e g r a v e y a r d ( ) c a l enda r ( ) bug z ill a ( ) R an k Product namebzbarsky bugzilla dbarontimeless matti nswer to RQ1 . The developer prioritization in products differs from that in the whole project. Specially, certain top developers in the whole project may contribute little or noth-ing to a product. Among the participated products, top de-velopers contribute much to most of products. C. Answer to RQ2, Evolution of Developer Prioritization
For a large scale software product, personnel changes are common in development. An active developer may be inac-tive in a period, or even retire [10]. In this sub-section, we investigate the evolution of the developer prioritization over time. Based on Table II, Eclipse has been developed for over 9 years and Mozilla has been for over 12 years. To investigate the evolution, we select bug reports in 9 continuous years from Jan. 2002 to Dec. 2010 (since Eclipse is started from Oct. 2001). We choose half a year as the unit time by follow-ing [17]. Thus, the time for each project is divided into 18 periods. We denote the first half of a year with “f” and the second half of a year with “s”. For each unit time, we gener-ate the developer prioritization in the whole projects and analyze the evolution of leading developers over time. We illustrate the changes of top developers in Fig. 4. To simplify the expressions, we denote the developers who are ranked in top 1000 in the whole project as leaders . Given a period, a developer is called an old leader or a new leader, if he or she is a leader in the last period or not, respectively. As shown in Fig. 4, the ratios in Mozilla are more stable than those in Eclipse. Moreover, for both projects, the changes after 2005 are stable. In Eclipse, the ratio of new developers who are also leaders is over 20% in each unit time. From 2002s to 2006s, the ratio of old developers who are also old leaders increases with time. In Mozilla, the ratio of new de-velopers who are leaders is less than 20%. In both Eclipse and Mozilla, the developer prioritization changes over time. We list two possible reasons for this fact. One is that the developer prioritization always changes be-cause of the complexity of such large projects; the other is the unit time in our experiments is not short enough to rec-ognize the fixed developer prioritization. A further study is needed to explore the appropriate unit time to model un-changed developer prioritization in projects.
Answer to RQ2 . The developer prioritization evolves over time. A new developer can join the projects and become a developer with high priorities. D. Answer to RQ3, Tolerance of Noisy Comments
The process of fixing and localizing bugs suffers from the bad quality of bug reports [18]. Noises in bug reports are common in bug repertories. For example, in the bug report with ID 1 of Eclipse, Comment 45 is just a test of a user ac-count, which has nothing to do with the content of the bug. Since the developer prioritization is built based on the bug comments, we investigate whether the developer prioritiza-tion is sensitive to noisy bug comments. Due to the lack of existing method to identify noisy comments, we label noisy comments with a heuristic. We consider two types of comments as noises, namely the com-ments by inexperienced developers and the comments writ-ten in very few words. Note that not all the comments in these two types are noisy, e.g., Comment 2 of Bug 250031 in Eclipse, only containing a full stop, can be viewed as a noisy comment while Comment 2 of Bug 250033, only containing a word “verified”, is a useful comment. In this paper, we directly treat the two types of bug reports as noisy comments without further identification. In Table III, we present the ratios of developers and comments for the data sets, which are generated by removing two types of comments. The sev-en columns denote three data sets by removing comments of (a) Eclipse (b) Mozilla
Figure 4.
Percentage for developers and leaders over time. The leaders (top 1000 developers) in each unit time is considered. The ratio of a new developer who is a leader, the ratio of an old developer who is a leader, and the ratio of an old developer who is new as a leader are labeled in black, light gray, and dark gray, respectively. (a) Eclipse (b) Mozilla
Figure 5.
Changes for the ranks of top 10 developers among the original data set and new data sets after removing noises. The selected developers are top 10 developers in the original data set.
TABLE Ⅲ R ATIOS OF D EVELOPERS AND C OMMENTS IN D ATA S ETS
Project
Ratio >15 comments >10 comments >5 comments
Original >3 words >6 words >9 words Eclipse
Developer (%)
Comment (%)
Mozilla
Developer (%)
Comment (%) f s f s f s f s f s f s f s f s f s R a t i o Timenew developer & leader old developer & leader old developer & new leader f s f s f s f s f s f s f s f s f s R a t i o Timenew developer & leader old developer & leader old developer & new leader > c o mm en t s > c o mm en t s > c o mm en t s o r i g i na l > w o r d s > w o r d s > w o r d s R an k Data setdaniel_megert darin.eclipsejohn_arthorne eclipsedavid_williams olivier_thomanndenis.roy webmastertod_creasey steve_northover > c o mm en t s > c o mm en t s > c o mm en t s o r i g i na l > w o r d s > w o r d s > w o r d s R an k Data setbzbarsky bugzilla dbarontimeless matti gervasa roc brendanneil nexperienced developers (5, 10, and 15), the original data set, and three data sets by removing comments written in few words (3, 6, and 9). We illustrate the changes of the developer prioritization in Fig. 5. We can find that our approach for the developer prioritization is tolerant to noisy comments. The change among each data set is inconspicuous. This fact coincides with our expectation since we always focus on the develop-ers with high priorities, who contribute many comments to bug repositories. The noisy comments removed in our exper-iments only affect the developers with few comments.
Answer to RQ3 . The developer prioritization in our work is insensitive to noisy comments. Therefore, the devel-oper prioritization can be used to handle the real data sets, which include noisy bug comments. V. L EVERAGING THE D EVELOPER P RIORITIZATION
In this section, we explore the results of leveraging the developer prioritization to assist the tasks in bug repositories. We answer RQ4 by examining the effects on three typical tasks, i.e., bug triage, severity identification, and reopened bug prediction. All these tasks have been addressed to im-prove the quality of software development. We select such three tasks since they cover various aspects of predictive tasks. In details, bug triage [1] is a multiple-class task based on bug repositories; severity identification [20] is a binary-class task based on bug repositories; and reopened bug pre-diction [30] is a binary-class task based on both bug reposito-ries and change log repositories. In this section, we combine the input or the output of a task with developer priorities obtained by the developer pri-oritization. By combining with the input, we add new fea-tures to the predictive model while by combining with the output, we update the results of a task. In Fig. 6, we briefly summarize the process for improving the results of the three tasks by combining the developer priorities. A. Bug Triage
Bug triage is a typical problem in software maintenance, which aims to predict a correct developer for a new-coming bug [1]. Traditionally, a human developer (also called triager ) assigns new bugs to candidate developers. Automatic ap-proaches for bug triage have been developed to reduce time and labor cost. Most of existing work models bug triage as text classification and improves the accuracy of bug triage based on the knowledge from bug repositories [1], [19], [38], [32]. In this paper, we consider improving bug triage with the developer prioritization obtained from bug repositories. For a predicted list of developers by a classifier, we rank these developers by the priorities. Thus, the developer prioritiza-tion is used to discriminate the developers with similar prob-abilities in the prediction. We evaluate the results of bug triage with the incremen-tal learning framework, in which we chronologically sort all the bug reports and divide these bug reports into 11 folds [6]. Thus, we perform experiments in 10 rounds. In each round, we generate the developer prioritization from the training set and combine the developer priorities with the predicted re-sults of the classifiers. In Fig. 7, we present the evaluation framework in our work. We validate our approach on the bugs from 200001 to 300000 for Eclipse and bugs from 400001 to 500000 for Mozilla. We follow the existing work [1], [9] to remove the non-fixed bug reports (only bug reports with the resolution “fixed” are left) and inactive developers (in our work, devel-opers who have fixed less than 50 bugs are removed). As a result, 49762 bug reports of Eclipse and 30609 bug reports of Mozilla are left as data sets. For each bug report, the title and the description are extracted as an input text while the devel-oper who has fixed this bug is extracted as a label for the classifier. We convert the bug reports into vector space mod-el by tokenizing the sentences into terms. We perform the techniques of removing stop words, stemming, and tf-idf (a weighted term-frequency approach [33]) to generate the final data sets. We evaluate the experiments with the accuracy of top- k predicted developers since a recommendation list is always employed [2]. The accuracy is calculated as based on a recommendation list with size k . We employ two typical classifiers, i.e., Naive Bayes (NB) and Supporting Vector Machine (SVM). These classifiers are implemented by Weka [33]. Figure 6.
Leveraging the developer prioritization to predict tasks around bug repositories. We use the developer prioritization to assist three tasks. A circle denotes an action of predicting.
Figure 7.
Bug triage combined with the developer prioritization based on the incremental learning. In Round r , the first r folds are used for training a classifier and generating the developer prioritization while the ( r +1)th fold is used for testing. The developer prioritization is combined with the result of testing to form a new developer list. DeveloperComments
Developer Prioritization
Combined Fixer Rankings
Candidate Fixers
SeverityCombined Features Reopened or not
Combined Metrics B ug R e po s i t o r y Bug ReportsBug ReportsBug Metrics
Task
Bug Triage
Task
Severity Identification
Task
Reopened Bug Prediction
DeveloperPriorities DeveloperPrioritiesTriaging IdentifyingPredictingDeveloperPriorities
Fold 1 Fold 2 Fold 3 Fold 11Training Testing
Developer Prioritization Combined Ranking
Training Testing
Developer Prioritization Combined Ranking
Training Testing
Developer Prioritization Combined Ranking
Fold 4 Fold 10 ... ...
Round 1Round 2Round 10
Bug Reports n the following of this sub-section, we present the exper-imental results on bug triage. To obtain a high accuracy, we examine several ways to generate the developer prioritization from bug comments in training sets. We consider this prob-lem on two dimensions, namely the source and the time peri-od of developer prioritization. On one hand, in RQ1, we have presented that the developer prioritization changes in differ-ent products. Thus, we generate the developer prioritization based on products, components, and products & components, respectively. On the other hand, in RQ2, we have verified that the developer prioritization in two close periods can be very similar. Thus, we consider two kinds of time periods for building the developer prioritization, i.e., the accumulative folds in the training set and the latest fold in the training set. Taken Round 2 in Fig. 7 as an example, Folds 1 and 2 are used to build the developer prioritization by choosing the accumulative folds while only the Fold 2 is used by choosing the last fold. To improve bug triage, we combine the product-based and the component-based developer prioritization with the predicted results of classifiers. Given a new bug report, we extract its product and its component. We combine the prob-abilities in prediction with the developer priorities, which match its product and its component. Formally, given a new bug report , its product , and its component , the final score is , where is the probability predicted for each developer by a classifier, is the score in -based developer prioritization, is the score in -based developer prioritization, is the maximum value in developers for normalization ( ). The score or is set to zero if a developer never appears in a product or a component. Then, we rank the developers by these final scores and select developers with highest scores as the final results. To obtain the rankings of top- k developers for a bug report, we use a classifier to predict developers and rank these developers by the combined final scores. Then, top- k developers in the new rankings are se-lected as the final predicted results. In our work, we examine the results of top-5 developers. In Fig. 8, we present the accuracy of the top-1 developer in each round. In both Eclipse and Mozilla, the approaches combined with the developer prioritization can obtain higher accuracy than directly using SVM. Most of the results based on the latest fold are better than those based on the accumu-lative folds. As a result, we observe that the recent changes of developers can be more helpful than the accumulative (a) Eclipse (b) Mozilla Figure 8.
Accuracy for the top-1 predicted developer in 10-round incremental learning. SVM denotes the accuracy only based on the classifier, P and C denote the approach combining with the developer prioritization based on products and components, respectively. A and L denotes the approach based on the developer proritization, which is extracted from the accumulative folds and the latest fold in the training set, respectively. A cc u r a cy Round
SVM P+A C+AP+C+A P+L C+LP+C+L A cc u r a cy Round
SVM P+A C+AP+C+A P+L C+LP+C+L
TABLE Ⅳ P REFORMANCE OF B UG T RIAGE ON E CLIPSE AND M OZILLA
Project Classifier
Size
Approach Accuracy in Each Round (%)
Average Accuracy Improvement
1 2 3 4 5 6 7 8 9 10 Eclipse SVM
Top 1
SVM
SVM+DP
SVM
SVM+DP
SVM
SVM+DP NB Top 1 NB NB+DP NB NB+DP NB NB+DP
Mozilla SVM
Top 1
SVM
SVM+DP
SVM
SVM+DP
SVM
SVM+DP NB Top 1 NB NB+DP NB NB+DP NB NB+DP hanges. This observation coincides with the existing work [1], [6] that the latest activity of a developer is representative. Among seven approaches in each sub-figure of Fig. 8, the approach combined with the developer prioritization based on products and components has the highest accuracy. In the rest of this sub-section, we choose such combination as the approach in the experiments. In Table IV, we present the results of bug triage by com-bining the developer priorities with the output of the classifi-er in Eclipse and Mozilla. SVM+DP and NB+DP denote the results, which are ranked by combining the developer priori-tization with the output of SVM and NB, respectively. We can find that for both of SVM and NB, the accuracy is im-proved when combining with the developer prioritization. The average improvement for SVM is around 10% while the average improvement for NB is only about 2%. The reason for such results is that the combination with the developer prioritization is based on the probability predicted by classi-fiers and SVM has stronger ability on discriminating differ-ent classes than NB. Thus, for NB, the ranks of developers may not change too much by combining the developer priori-tization. As shown in Table IV, the improvement for the top 1 is larger than that for top 5. The improvement for SVM in some rounds are over 15%, e.g., Round 10 with top 3 devel-opers in Eclipse and Round 9 with top 3 in Mozilla. B. Severity Identification
Since the number of daily bugs is large to handle, new bugs in bug repositories are divided into different severities to process for different goals. Existing work (robotic severity [23], security severity [14], and critical severity [20]) has identified bug severities with predictive techniques. In this sub-section, we address the critical severity prob-lem. Lamkanfi et al. [20] predict whether a new bug is severe by adapting a text classification technique. In their work, a set of bug reports, including non-severe or severe ones, are divided into a training set and a test set. The 10-fold cross validation [33] is used to evaluate the results of classification. For a bug report with a self-reported severity, trivial or mi-nor , is considered as a non-severe bug while major , critical , or blocker is considered as a severe one. Given a new bug report, the title of this bug is extracted to build the vector space model. The stop word removal, stemming techniques are performed (note that no tf-idf technique is used). Based on the vector space of training sets, Naive Bayes (NB) classi-fier is employed to identify whether a bug is severe or not. In existing work by Lamkanfi et al. [20], the severity of a bug is predicted by the numeric vector, which is converted from the bug title. We consider adding the factors of devel-opers to predict severities. For new bugs, the priorities of reporters may provide more information to identify the se-verity, e.g., a reporter with a high priority may pay more attentions to the severe bugs. Therefore, for each bug report, we add two features from the developer prioritization to the original numeric vectors. These two features are two priority scores of reporters based on the products and components, respectively. Then the predictive vector has two more nu-meric features. Since values in predictive vectors may be imbalance, we normalize all the features to the range from 0 to 1 for each training set. In our work, we extract the data sets from bug reposito-ries of Eclipse (bugs 1-300000) and Mozilla (bugs 300001-600000). For each project, we select severe and non-severe bugs in three components in accordance with [20]. These data sets are presented in Table V. We evaluate the severity identification with precision , recall , and F-measure . We present the experimental results in Table VI. For each component, the first row denotes the classification based on the original vectors, which are extracted from bug titles, while the second row denotes the classification based on the mixed vectors, which are formed by adding two de-veloper scores. A value in bold denotes a result, which is over 1% better than the other result on the same component. As shown in Table VI, the prediction based on the mixed vectors can obtain better precision, recall, and F-measure on bugs in Mozilla while the difference of results in Eclipse is not significant. One possible reason for these results is the number of original numeric features is large and the effects of two new dimensions are not obvious to the predictive model. C. Reopened Bug Prediction
Reopened bug prediction aims to identify a bug report, which will be incorrectly fixed in the future. To our knowledge, only Shihab et al. [30] have proposed the method to predict a reopened bug. In their work, they extract 22 fac-tors in 4 dimensions to build the predictive model based on AdaBoost [33]. Among the 4 dimensions of factors, one di-mension is called “people dimension”, which consists of 4 factors about the developer information of a bug report, i.e., the reporter name, the fixer name, the reporter experience, and the fixer experience. In this paper, we do not discuss the improvement of the predictive model of reopened bug prediction. We only con-
TABLE Ⅴ N UMBER OF B UGS FOR E ACH S EVERITY
Project Product: Component
Platform: UI 1425 3284 JDT: UI 1425 1533 JDT: Text 816 558
Mozilla
Core: Layout 255 929 Camino: Bookmarks 62 40 Firefox: General 2658 10124
TABLE Ⅵ P ERFORMANCE ON S EVERITY I DENTIFICATION
Project
Product: Component
Non-severe bugs (%) Severe bugs (%)
Precision Recall F-measure Precision Recall F-measure
Eclipse
Platform: UI
JDT: UI
JDT: Text
Mozilla
Core: Layout
Camino: Bookmarks
Firefox: General ider whether our developer prioritization can provide simi-lar factors to this model. We evaluate the experiments with the 10-fold cross validation on the benchmark data set in [30]. This data set consists of 1530 bugs (246 are reopened bugs and 1284 are not), which are extracted from bugs 1-300000 in Eclipse. We generate the developer prioritization (without specifying a product or a component) on the bug comments of bugs 1-300000 and extract the scores of reporters and fix-ers, who are specified in the data set. Note that we have not conducted experiments on Mozilla since only the benchmark for Eclipse is shared in [30]. In the experiments, we add two factors to the people di-mension, namely the reporter priority score and the fixer priority score. These 6 factors can be divided into three groups, i.e., name , experience , and priority . We list 4 combi-nations of these three groups of factors in the column “factor” in Table VII (Exp. is short for the group of factors experi-ence ). We run the algorithm AdaBoost in [30] to examine the results. As shown in Table VII, by changing the original factors of name and experience with priority , the performance is improved. And the combination of all the three groups can provide the highest performance. We summarize the results in Table VII that the factors based on the developer prioriti-zation can provide similar features to the model for predict-ing reopened bugs and can slightly improve the existing re-sults. Note that the improvement by adding the factors of priority scores is not significant. This fact suffers from two possible reasons. One is the small number of training set may limit the predictive ability of factors; the other is the existing results may be good enough and hard to improve. Answer to RQ4 . By examining three typical tasks in bug repositories, we conclude that the developer prioritization is helpful to improve the results of these tasks, especially, the task of bug triage. VI. T HREATS TO V ALIDITY A. Building the Developer Prioritization
In our work, we build the developer prioritization from bug comments in bug repositories. The developer rankings are obtained to denote the priorities of developers. Since there is no real ranking of developers in software projects, it is hard to validate whether our obtained rankings are coinci-dent with the real collaboration in development. To address this problem, a good way is to conduct a case study to exam-ine the differences between the developer prioritization and the collaboration in development. Moreover, the developer prioritization in our work can enhance simple measures, e.g., distinguishing the developers with the same out-degree. An experiment should be conducted to compare the priorities between our work and simply measuring out-degrees. Besides the developer prioritization for a whole project, we have also proposed product-based and component-based developer prioritization in our work. We extract the products and components from the self-reported terms of bug reports. However, such terms of products or components may mis-match the real terms since reporters lack experiences to iden-tify the correct products or components [40]. To completely avoid such mismatch, a technique for identifying the correct product and components should be developed. B. Analysis of Developer Prioritization
In Section IV, we explore the evolution of developer pri-oritization over time. The time period is divided into several half years to study the changes in the process of development. Besides the time-based evolution, version-based evolution of the developer prioritization may provide more information. We do not investigate version-based evolution in our work since the bug reports in different versions are hard to collect. For example, among bugs 200001-300000, only 7450 bugs (7.45%) are identified with version information, which be-long to 38 products. To analyze the noise tolerance of developer prioritization, we recognize noisy comments with a heuristic, which views the comments by inactive developers and the comments with few words as noises. Many of such comments consist of noisy information, but some meaningful comments may also be viewed as noises, such as comments with only one word “fixed” or “verified”. To exactly recognize noisy comments, manually labeling is more helpful than a heuristic. C. Assisting Software Tasks
In this paper, we show that the developer prioritization is effective to improve the tasks in bug repositories based on the empirical evaluation. The developer priorities can add more information to the input features or update the output of classifiers. However, further questions may be proposed, for example, what is the internal relationship between the devel-oper prioritization and the goal of a task? And why is the social behavior of developers helpful? These questions are not easy to answer. In this paper, the developer prioritization can build a bridge from bug repositories to predictive tasks. For further work, a systematical case study can provide more information to explore the correlation between the developer prioritization and the predictive tasks. VII. R ELATED W ORK A. Social Network Analysis in Software Repositories
Bird et al. [3] mine social networks from email achieves and analyze the developer activity based on social network measures. Their later work [8] explores communities from the social networks, which are representative of the collabo-ration of developer behavior. Wolf et al. [34] employ the team communication network to predict the failures of soft-ware builds. In their work, the centrality measures in social networks are extracted as features in the predictive model.
TABLE Ⅶ F ACTORS ON R EOPENED B UG P REDICTION IN E CLIPSE
Factor
Reopened bugs (%) Non-reopened bugs (%)
Accuracy
Precision Recall
F-measure
Precision Recall
F-measure ( Original )Name+Exp.
Exp.+Priority
Name+Priority
Name+Exp.+Priority ocial networks have been proposed to improve the software failure prediction. To predict failures, Meneely et al. [25] construct developer networks on change log repositories and Pinzger et al. [27] build developer-module networks on binary repositories. Bird et al. [7] combine the topological properties with social networks and investigate multiple types of relationships to predict failures. Moreover, Bettenburg & Hassan [4] study the impacts of socio-technical measures through the failure detection. Since the social techniques are effective for indicating software fail-ures, Meneely & Williams [24] empirically validate that the social network metrics can represent the collaboration rela-tionship in software development. By building a social networking service for developers, Begel et al. [5] introduce a Codebook framework to discover the inter-team coordination in development. They have con-ducted two applications to evaluate the effectiveness of their framework. In bug repositories, Hong et al. [17] have examined the developer social networks in bug repositories. They discover the sub-communities of a developer network and investigate the evolution over time. In this paper, we also focus on the socio-technical analysis on bug repositories. In contrast to the community discovery in [17], we explore the developer prioritization in bug repositories. We analyze the developer rankings and improve three existing tasks in bug repositories. B. Bug Repositories
In this paper, we investigate the developer prioritization in bug repositories. In existing work on bug repositories, Fischer et al. [13] explore the proximity of software features. In their work, bug report analysis is used to study and visual-ize the relations between software features. The quality of bug reports is important for locating and fixing bugs. Hooimeijer & Weimer [18] present the first work to model the quality of bug reports. Bettenburg et al. [9] point out that duplicate bugs contain extra information for bug fixing, which are useful to improve bug triage. Zim-mermann et al. [40] study the evidence for the mismatch between developer expectation and bug reports based on a systematic questionnaire survey. Xiao & Afzal [36] propose a search-based approach for the resource scheduling on bug fixing tasks. Most work on bug repositories treats developers and bugs separately. Our previous work [37] proposes an integrated view of developers and bug reports. We transfer the interac-tions between developers and bug reports to requirements engineering to supplement the lack of open requirements. In this paper, we extract the developer prioritization in bug repositories. Besides the multiple aspects of analysis, we leverage the developer prioritization to predict software tasks. C. Predicting Tasks in Bug Repositories
The goal of bug triage is to automatically assign a new bug to the correct developer to avoid the expensive cost of maintenance. Čubranić & Murphy [10] have proposed the first work of bug triage, which transforms bug triage to a text categorization problem. Anvik et al. [1] extend the above work with a recommendation list and multiple classifiers. Jeong et al. [19] and Bhattacharya & Neamtiu [6] propose a tossing graph based approach to improve bug triage with the previous assignment history of bug reports. Our previous work [38] proposes a semi-supervised learning approach to avoid the lack of qualified bug reports. Recent work by Anvik & Murphy [2] investigates the effects of recommend-ers to assist bug triage for streamlining the development pro-cess. Other work also addresses the problem of bug triage, such as the training set reduction [39], the fuzzy-set and cache-based approach [32], and the cost-aware bug triage [26]. Severity identification is to detect the bug severities to guide the resource allocation and planning of bug fixing. To date, three types of severities are studied. Menzies & Macus [23] first propose a text mining approach to detect the 5-level robotic severity for bug reports in NASA databases. Gegick et al. [14] and Lamkanfi et al. [20] have further predicted the security severity and the critical severity of bug reports. Reopened bug prediction is to detect whether a bug is fixed in a correct way. Shihab et al. [30] study and predict reopened bugs on 22 factors in four dimensions, which are extracted from both bug repositories and source code reposi-tories. In an empirical study on characterizing which bugs get fixed, Guo et al. [15] present that the times of reopenings is a factor to indicate whether a bug can be fixed. In this paper, we empirically evaluate whether the devel-oper prioritization can improve the results of the above three tasks. Besides the three mentioned tasks, existing work im-proves the software quality in bug repositories on other tasks. For example, Rastkar et al. [29] summarize long bug reports to avoid redundancies and noises in bug repositories; Runeson et al. [28], Wang et al. [35], and Sun et al. [31] de-tect duplicate bug reports to reduce the expense for handling bugs in large scale repositories. VIII. C ONCLUSION AND F UTURE W ORK
In this paper, we model the developer prioritization in bug repositories by extending a socio-technical approach. We analyze three problems of the developer prioritization, namely the characteristics in products, the evolution, and the tolerance of noises. Based on the analysis, we investigate the ways to leverage the developer prioritization to improve three typical tasks in bug repositories. The results are studied on over 900000 bug reports in Eclipse and Mozilla. Our future work is to investigate a task-based developer prioritization in bug repositories to improve a specified task with the developer rankings. In contrast to the general model of the developer prioritization, we want to provide a model to add more knowledge to handle the problems in a specified task. For example, fixers of bug reports should be added more weights in the developer prioritization to improve bug triage. A
CKNOWLEDGMENT
We greatly thank the anonymous reviewers for their in-sightful comments. This work is partially supported by the National Natural Science Foundation of China under grants 1175062, 60805024, and 61033012, and the “Software + X” funding of Dalian University of Technology. R
EFERENCES [1]
J. Anvik, L. Hiew, and G.C. Murphy, “Who Should Fix This Bug?,” Proc. 28th Intl. Conf. Software Engineering (ICSE ’06), May 2006, pp. 361-370. [2]
J. Anvik and G.C. Murphy, “Reducing the Effort of Bug Report Triage: Recommenders for Development-Oriented Decisions,” ACM Trans. Software Engineering & Methodology, vol.20, no.3,Aug. 2011. [3]
C. Bird, A. Gourley, P. Devanbu, M. Gertz and A. Swaminathan, “Mining Email Social Networks,” Proc. 3rd Intl. Workshop Mining Software Repositories (MSR ’06), May 2006, pp.137-143. [4]
N. Bettenburg and A.E. Hassan, “Studying the Impact of Social Structures on Software Quality,” Proc. IEEE 18th Intl. Conf. Program Comprehension (ICPC ’10), Jun. 2010, pp.124-133. [5]
A. Begel, Y.P. Khoo, and T. Zimmermann, “Codebook: Discovering and Exploiting Relationships in Software Repositories,” Proc. 32nd Intl. Conf. Software Engineering (ICSE ’10), May 2010, pp. 125-134. [6]
P. Bhattacharya and I. Neamtiu, “Fine-Grained Incremental Learning and Multi-Feature Tossing Graphs to Improve Bug Triaging,” Proc. 26th IEEE Intl. Conf. Software Maintenance (ICSM ’10), Sept. 2010, pp. 1-10. [7]
C. Bird, N. Nagappan, H. Gall, B. Murphy, and P. Devanbu, “Putting it All Together: Using Socio-Technical Networks to Predict Failures,” Proc. 20th Intl. Symp. Software Reliability Engineering (ISSRE ’09), Nov. 2009, pp.109-119. [8]
C. Bird, D. Pattison, R. D’Souza, V. Filkov and P. Devanbu, “Latent Social Structure in Open Source Projects,” Proc. 16th ACM SIGSOFT Intl. Symp. Foundations of software engineering (FSE ’08), Nov. 2008, pp.24-35. [9]
N. Bettenburg, R. Premraj, T. Zimmermann, and S. Kim, “Duplicate Bug Reports Considered Harmful… Really?,” Proc. 24th IEEE Intl. Conf. Software Maintenance (ICSM ’08), Sept. 2008, pp. 337-345. [10]
D. Čubranić and G.C. Murphy, “Automatic Bug Triage Using Text Categorization,” Proc. 16th Intl. Conf. Software Engineering & Knowledge Engineering (SEKE ’04), Jun. 2004, pp. 92-97. [11]
T. DeMarco and T. Lister, Peopleware: Productive Projects and Teams, 2nd ed. Dorset House, New York, 1999. [12]
Eclipse. http://eclipse.org/. [13]
M. Fischer, M. Pinzger and H. Gall, “Analyzing and Relating Bug Report Data for Feature Tracking,” Proc. 10th Working Conf. Reverse Engineering (WCRE ’03), Nov. 2003, pp. 90-101. [14]
M. Gegick, P. Rotella, and T. Xie, “Identifying Security Bug Reports via Text Mining: An Industrial Case Study,” Proc.7th IEEE Working Conf. Mining Software Repositories (MSR ’10), May 2010, pp. 11-20. [15]
P.J. Guo, T. Zimmermann, N. Nagappan, and B. Murphy, “Characterizing and Predicting Which Bugs Get Fixed: An Empirical Study of Microsoft Windows,” Proc. 32nd Intl. Conf. Software Engineering (ICSE ’10), May 2010, pp. 495-504. [16]
A.E. Hassan, “The Road Ahead for Mining Software Repositories,” Proc. Frontiers of Software Maintenance (FoSM ’08), Sept. 2008, pp. 48-57. [17]
Q. Hong, S. Kim, S.C. Cheung, and C. Bird, “Understanding a Developer Social Network and its Evolution,” Proc. 27th IEEE Intl. Conf. Software Maintenance (ICSM ’11), Sept. 2011, pp. 323-332. [18]
P. Hooimeijer and W. Weimer, “Modeling Bug Report Quality,” Proc. 22nd IEEE/ACM Intl. Conf. Automated Software Engineering (ASE ’07), Nov. 2007, pp. 34-43. [19]
G. Jeong, S. Kim, and T. Zimmermann, “Improving Bug Triage with Tossing Graphs,” Proc. 17th ACM SIGSOFT Symp. Foundations of Software Engineering (FSE ’09), Aug. 2009, pp. 111-120. [20]
A. Lamkanfi, S. Demeyer, E. Giger, and B. Goethals, “Predicting the Severity of a Reported Bug,” Proc. 7th IEEE Working Conf. Mining Software Repositories (MSR ’10), May 2010, pp. 1-10. [21]
L. Lü, Y.-C. Zhang, C.H. Yeung, and T. Zhou, “Leaders in Social Networks, the Delicious Case,” PLoS One, vol. 6, no. 6, Jun. 2011. [22]
Mozilla. http://mozilla.org/. [23]
T. Menzies and A. Marcus, “Automated Severity Assessment of Software Defect Reports,” Proc. IEEE Conf. Software Maintenance (ICSM ’08), Sept. 2008, pp. 346-355. [24]
A. Meneely and L. Williams, “Socio-Technical Developer Networks: Should We Trust Our Measurements?,” Proc. 33rd Intl. Conf. Software Engineering (ICSE ’11), May 2011, pp. 281-290. [25]
A. Meneely, L. Williams, W. Snipes, and J. Osborne, “Predicting Failures with Deverloper Networks and Social Network Analysis,” Proc. 16th ACM SIGSOFT Intl. Symp. Foundations of Software Engineering (FSE ’08), Nov. 2008, pp. 13-23. [26]
J.-W. Park, M.-W. Lee, J. Kim, S.-W. Hwang, and S. Kim, “CosTriage: A Cost-Aware Triage Algorithm for Bug Reporting Systems,” Proc. 25th Conf. Artificial Intelligence (AAAI ’11), Aug. 2011, pp. 139-144. [27]
M. Pinzger, N. Nagappan, and B. Murphy, “Can Developer-Module Networks Predict Failures?,” Proc. 16th ACM SIGSOFT Intl. Symp. Foundations of Software Engineering (FSE ’08), Nov. 2008, pp. 2-12. [28]
P. Runeson, M. Alexandersson, and O. Nyholm, “Detection of Duplicate Defect Reports Using Natural Language Processing,” Proc. Intl. Conf. Software Engineering (ICSE ’07), May 2007, pp. 499-510. [29]
S. Rastkar, G.C. Murphy, and G. Murray, “Summarizing Software Artifacts: A Case Study of Bug Reports,” Proc. 32nd Intl. Conf. Software Engineering (ICSE ’10), May 2010, pp. 505-514. [30]
E. Shihab, A. Ihara, Y. Kamei, W.M. Ibrahim, M. Ohira, B. Adams, A.E. Hassan, and K. Matsumoto, “Predicting Re-opened Bugs: A Case Study on the Eclipse Project,” Proc. 17th Working Conf. Reverse Engineering (WCRE ’10), Oct. 2010, pp. 249-258. [31]
C. Sun, D. Lo, X. Wang, J. Jiang, and S.-C. Khoo, “A Discriminative Model Approach for Accurate Duplicate Bug Report Retrieval,” Proc. 32nd Intl. Conf. Software Engineering (ICSE ’10), May 2010, pp. 45-54. [32]
A. Tamrawi, T.T. Nguyen, J.M. Al-Kofahi, and T.N. Nguyen, “Fuzzy-Set and Cache-Based Approach for Bug Triaging,” Proc. 19th ACM SIGSOFT Symp. Foundations of Software Engineering (FSE ’11), Sept. 2011, pp. 365-375. [33]
I.H. Witten, E. Frank, and M.A. Hall, Data Mining: Practical Machine Learning Tools and Techniques, 3rd ed. Morgan Kaufmann, Burlington, MA, 2011. [34]
T. Wolf, A. Schröter, D. Damian, and T. Nguyen, “Predicting Build Failures Using Social Network Analysis on Developer Communication,” Proc. 31st Intl. Conf. Software Engineering (ICSE ’09), May 2009, pp. 1-11. [35]
X. Wang, L. Zhang, T. Xie, J. Anvik, and J. Sun, “An Approach to Detecting Duplicate Bug Reports Using Natural Language and Execution Information,” Proc. 30th Intl. Conf. Software Engineering (ICSE ’08), May 2008, pp. 461-470. [36]
J. Xiao and W. Afzal, “Search-based Resource Scheduling for Bug Fixing Tasks,” Proc. 2nd Intl. Symp. Search Based Software Engineering (SSBSE ’10), Sept. 2010, pp. 133-142. [37]
J. Xuan, H. Jiang, Z. Ren, Z. Luo, “Solving the Large Scale Next Release Problem with a Backbone Based Multilevel Algorithm,” IEEE Trans. Software Engineering, preprint, doi: 10.1109/TSE.2011.92. [38]
J. Xuan, H. Jiang, Z. Ren, J. Yan, and Z. Luo, “Automatic Bug Triage Using Semi-Supervised Text Classification,” Proc. 22th Intl. Conf. Software Engineering & Knowledge Engineering (SEKE ’10), Jul. 2010, pp. 209-214. [39]