Morakot Choetkiertikul

Predicting Delays in Software Projects Using Networked Classification (T)

Software projects have a high risk of cost and schedule overruns, which has been a source of concern for the software engineering community for a long time. One of the challenges in software project management is to make reliable prediction of delays in the context of constant and rapid changes inherent in software projects. This paper presents a novel approach to providing automated support for project managers and other decision makers in predicting whether a subset of software tasks (among the hundreds to thousands of ongoing tasks) in a software project have a risk of being delayed. Our approach makes use of not only features specific to individual software tasks (i.e. local data) -- as done in previous work -- but also their relationships (i.e. networked data). In addition, using collective classification, our approach can simultaneously predict the degree of delay for a group of related tasks. Our evaluation results show a significant improvement over traditional approaches which perform classification on each task independently: achieving 46% -- 97% precision (49% improved), 46% -- 97% recall (28% improved), 56% -- 75% F-measure (39% improved), and 78% -- 95% Area Under the ROC Curve (16% improved).

Characterization and prediction of issue-related risks in software projects

Identifying risks relevant to a software project and planning measures to deal with them are critical to the success of the project. Current practices in risk assessment mostly rely on high-level, generic guidance or the subjective judgements of experts. In this paper, we propose a novel approach to risk assessment using historical data associated with a software project. Specifically, our approach identifies patterns of past events that caused project delays, and uses this knowledge to identify risks in the current state of the project. A set of risk factors characterizing “risky” software tasks (in the form of issues) were extracted from five open source projects: Apache, Duraspace, JBoss, Moodle, and Spring. In addition, we performed feature selection using a sparse logistic regression model to select risk factors with good discriminative power. Based on these risk factors, we built predictive models to predict if an issue will cause a project delay. Our predictive models are able to predict both the risk impact (i.e. the extend of the delay) and the likelihood of a risk occurring. The evaluation results demonstrate the effectiveness of our predictive models, achieving on average 48%-81% precision, 23%-90% recall, 29%-71% F-measure, and 70%-92% Area Under the ROC Curve. Our predictive models also have low error rates: 0.39-0.75 for Macro-averaged Mean Cost-Error and 0.7-1.2 for Macro-averaged Mean Absolute Error.

Predicting the delay of issues with due dates in software projects

Issue-tracking systems (e.g. JIRA) have increasingly been used in many software projects. An issue could represent a software bug, a new requirement or a user story, or even a project task. A deadline can be imposed on an issue by either explicitly assigning a due date to it, or implicitly assigning it to a release and having it inherit the release’s deadline. This paper presents a novel approach to providing automated support for project managers and other decision makers in predicting whether an issue is at risk of being delayed against its deadline. A set of features (hereafter called risk factors) characterizing delayed issues were extracted from eight open source projects: Apache, Duraspace, Java.net, JBoss, JIRA, Moodle, Mulesoft, and WSO2. Risk factors with good discriminative power were selected to build predictive models to predict if the resolution of an issue will be at risk of being delayed. Our predictive models are able to predict both the the extend of the delay and the likelihood of the delay occurrence. The evaluation results demonstrate the effectiveness of our predictive models, achieving on average 79 % precision, 61 % recall, 68 % F-measure, and 83 % Area Under the ROC Curve. Our predictive models also have low error rates: on average 0.66 for Macro-averaged Mean Cost-Error and 0.72 Macro-averaged Mean Absolute Error.

Who Will Answer My Question on Stack Overflow

Stack Overflow is a highly successful Community Question Answering (CQA) service for software developers with more than three millions users and more than ten thousand posts per day. The large volume of questions makes it difficult for users to find questions that they are interested in answering. In this paper, we propose a number of approaches to predict who will answer a new question using the characteristics of the question (i.e. Topic) and users (i.e. Reputation), and the social network of Stack Overflow users (i.e. Interested in the same topic). Specifically, our approach aims to identify a group of users (candidates) who have the potential to answer a new question by using feature-based prediction approach and social network based prediction approach. We develop predictive models to predict whether an identified candidate answers a new question. This prediction helps motivate the knowledge exchanging in the community by routing relevant questions to potential answerers. The evaluation results demonstrate the effectiveness of our predictive models, achieving 44% precision, 59% recall, and 49% F-measure (average across all test sets). In addition, our candidate identification techniques can identify the answerers who actually answer questions up to 12.8% (average across all test sets).

Predicting Delivery Capability in Iterative Software Development

Iterative software development has become widely practiced in industry. Since modern software projects require fast, incremental delivery for every iteration of software development, it is essential to monitor the execution of an iteration, and foresee a capability to deliver quality products as the iteration progresses. This paper presents a novel, data-driven approach to providing automated support for project managers and other decision makers in predicting delivery capability for an ongoing iteration. Our approach leverages a history of project iterations and associated issues, and in particular, we extract characteristics of previous iterations and their issues in the form of features. In addition, our approach characterizes an iteration using a novel combination of techniques including feature aggregation statistics, automatic feature learning using the Bag-of-Words approach, and graph-based complexity measures. An extensive evaluation of the technique on five large open source projects demonstrates that our predictive models outperform three common baseline methods in Normalized Mean Absolute Error and are highly accurate in predicting the outcome of an ongoing iteration.

A Risk Assessment Tool Using a CMMI Quantitative Approach

 Abstract—A notable purpose of offshoring is to gain competitive advantages in software development business. While offshoring offers great opportunities, it also creates a new trend of threats in software development due to differences in culture, languages, time zones, and development processes deployed. Maximizing the profits of the offshoring relies on strong remote project management skills including remote risk analysis. It is important that potential risks should be detected promptly and there should be quantitative indicators that can help to precisely monitor risks in a remote manner. In that view, we present an offshoring risk assessment tool which uses our formally defined CMMI quantitative approach. This tool is implemented on IBM JAZZ platform to enable quantitative risk monitoring and assessment for application life cycle support.

A CMMI-Based Automated Risk Assessment Framework

Risk assessment is crucial to the increase of software development project success. Current risk assessment approaches provide only a rough guide. Risk assessment experts and domain experts are required in conducting risk assessments in software projects. Therefore, traditional risk assessment approaches require extra activities besides development tasks, and possibly leading to extra costs. We believe that an effective risk assessment approach should be transparently embedded in software development process. This paper aims to present an automated risk assessment framework using CMMI and risk taxnomy as a guidance to develop a risk assessment model. A pragmatic approach will be applied as a basis in building this suggested risk prediction model and the case studies of our practice. These studies are considered as our proof of concept.

Poster: Predicting Components for Issue Reports Using Deep Learning with Information Retrieval

Assigning an issue to the correct component(s) is challenging, especially for large-scale projects which have are up to hundreds of components. We propose a prediction model which learns from historical issues reports and recommends the most relevant components for new issues. Our model uses the deep learning Long Short-Term Memory to automatically learns semantic features representing an issue report, and combines them with the traditional textual similarity features. An extensive evaluation on 142,025 issues from 11 large projects shows our approach outperforms alternative techniques with an average 60% improvement in predictive performance.

Threshold-based prediction of schedule overrun in software projects

Risk identification is the first critical task of risk management for planning measures to deal with risks. While, software projects have a high risk of schedule overruns, current practices in risk management mostly rely on high level guidance and the subjective judgements of experts. In this paper, we propose a novel approach to support risk identification using historical data associated with a software project. Specifically, our approach identifies patterns of abnormal behaviours that caused project delays and uses this knowledge to develop an interpretable risk predictive model to predict whether current software tasks (in the form of issues) will cause a schedule overrun. The abnormal behaviour identification is based on a set of configurable threshold-based risk factors. Our approach aims to provide not only predictive models, but also an interpretable outcome that can be inferred as the patterns of the combinations between risk factors. The evaluation results from two case studies (Moodle and Duraspace) demonstrate the effectiveness of our predictive models, achieving 78% precision, 56% recall, 65% F-measure, 84% Area Under the ROC Curve.

Predicting Issues for Resolving in the Next Release

Deciding which features or requirements (or commonly referred to as issues) to be implemented for the next release is an important and integral part of any type of incremental development. Existing approaches consider the next release problem as a single or multi-objective optimization problem (on customer values and implementation costs) and thus adopt evolutionary search-based techniques to address it. In this paper, we propose a novel approach to the next release problem by mining historical releases to build a predictive model for recommending if a requirement should be implemented for the next release. Results from our experiments performed on a dataset of 22,400 issues in five large open source projects demonstrate the effectiveness of our approach.