Guenther Ruhe

A flexible method for software effort estimation by analogy

Effort estimation by analogy uses information from former similar projects to predict the effort for a new project. Existing analogy-based methods are limited by their inability to handle non-quantitative data and missing values. The accuracy of predictions needs improvement as well. In this paper, we propose a new flexible method called AQUA that is able to overcome the limitations of former methods. AQUA combines ideas from two known analogy-based estimation techniques: case-based reasoning and collaborative filtering. The method is applicable to predict effort related to any object at the requirement, feature, or project levels. Which are the main contributions of AQUA when compared to other methods? First, AQUA supports non-quantitative data by defining similarity measures for different data types. Second, it is able to tolerate missing values. Third, the results from an explorative study in this paper shows that the prediction accuracy is sensitive to both the number N of analogies (similar objects) taken for adaptation and the threshold T for the degree of similarity, which is true especially for larger data sets. A fixed and small number of analogies, as assumed in existing analogy-based methods, may not produce the best accuracy of prediction. Fourth, a flexible mechanism based on learning of existing data is proposed for determining the appropriate values of N and T likely to offer the best accuracy of prediction. New criteria to measure the quality of prediction are proposed. AQUA was validated against two internal and one public domain data sets with non-quantitative attributes and missing values. The obtained results are encouraging. In addition, acomparative analysis with existing analogy-based estimation methods was conducted using three publicly available data sets that were used by these methods. Intwo of the three cases, AQUA outperformed all other methods.

Adopting GQM based measurement in an industrial environment

Schlumberger RPS (Retail Petroleum Systems) integrated the Goal/Question/Metric approach into their existing measurement programs to improve their program performance. Key to their success was the use of feedback sessions as a forum to analyze and interpret measurement data. The paper discusses the elements of the GQM approach and fitting GQM to a measurement program.

The Cognitive Process of Decision Making

Decision making is one of the basic cognitive processes of human behaviors by which a preferred option or a course of actions is chosen from among a set of alternatives based on certain criteria. Decision theories are widely applied in many disciplines encompassing cognitive informatics, computer science, management science, economics, sociology, psychology, political science, and statistics. A number of decision strategies have been proposed from different angles and applica-tion domains such as the maximum expected utility and Bayesian method. However, there is still a lack of a fundamental and mathematical decision model and a rigorous cognitive process for decision making. This article presents a fundamental cognitive decision making process and its mathematical model, which is described as a sequence of Cartesian-product based selections. A rigorous description of the decision process in real-time process algebra (RTPA) is provided. Real-world decisions are perceived as a repetitive application of the fundamental cognitive process. The result shows that all categories of decision strategies fit in the formally described decision process. The cognitive process of decision making may be applied in a wide range of decision-based systems such as cognitive informatics, software agent systems, expert systems, and decision support systems.

A systematic approach for solving the wicked problem of software release planning

Release planning is known to be a cognitively and computationally difficult problem. Different kinds of uncertainties make it hard to formulate and solve the problem. Our solution approach called EVOLVE+ mitigates these difficulties by (i) an evolutionary problem solving method combining rigorous solution methods to solve the actual formalization of the problem combined with the interactive involvement of the human experts in this process, (ii) provision of a portfolio of diversified and qualified solutions at each iteration of the solution process, and (iii) the application of a multi-criteria decision aid method (ELECTRE IS) to assist the selection of the final solution from a set of qualified solutions. At the final stage of the process, an outranking relation is established among the qualified candidate solutions to address existing soft constraints or objectives. A case study is provided to illustrate and initially evaluate the given approach. The proposed method and results are not limited to software release planning, but can be adapted to a wider class of wicked planning problems.

Supporting Software Release Planning Decisions for Evolving Systems

Large-scale software systems constantly change during system evolution for feature enhancement. Most of the features originate from diverse stakeholders that require their needs to be met despite resource and risk constraints. In such large systems, the number of features requested during the different releases of the system typically exceeds the available resources. Release planning involves decision making about what new features or changes to implement during which release of the software. Existing release planning techniques are not targeted at evolving systems; in this case, knowledge about existing software product is core to making meaningful release decisions. In this paper, we describe ten key technical and nontechnical aspects impacting release planning. Based on these aspects, we evaluate seven existing release planning methods. We have also proposed a new release planning framework that considers the effect of existing system characteristics on release planning decisions. Initial realization of this framework focuses on historical defect data to characterize the health of system components. This proposed approach extends the existing solution method called EVOLVE* by (i) the proactive analysis of the risk involved in integrating new features into existing components of the system and (ii) identifying the importance of estimating the integration effort for each feature based on system characteristics. An illustrative example is also presented

Impact Analysis of Missing Values on the Prediction Accuracy of Analogy-based Software Effort Estimation Method AQUA

Effort estimation by analogy (EBA) is often confronted with missing values. Our former analogy- based method AUQA is able to tolerate missing values in the data set, but it is unclear how the percentage of missing values impacts the prediction accuracy and if there is an upper bound for how big this percentage might become in order to guarantee the applicability of AQUA. This paper investigates these questions through an impact analysis. The impact analysis is conducted for seven data sets being of different size and having different initial percentages of missing values. The major results are that (i) we confirm the intuition that the more missing values, the poorer the prediction accuracy of AQUA; (ii) there is a quadratic dependency between the prediction accuracy and the percentage of missing values; and (Hi) the upper limit of missing values for the applicability of AQUA is determined as 40%. These results are obtained in the context of AQUA. Further analysis is necessary for other ways of applying EBA, such as using different similarity measures or analogy adaptation methods from those used in AQUA. For that purpose, the experimental design in this study can be adapted.

Analysis of attribute weighting heuristics for analogy-based software effort estimation method AQUA+

Estimation by analogy (EBA) predicts effort for a new project by aggregating effort information of similar projects from a given historical data set. Existing research results have shown that a careful selection and weighting of attributes may improve the performance of the estimation methods. This paper continues along that research line and considers weighting of attributes in order to improve the estimation accuracy. More specifically, the impact of weighting (and selection) of attributes is studied as extensions to our former EBA method AQUA, which has shown promising results and also allows estimation in the case of data sets that have non-quantitative attributes and missing values. The new resulting method is called AQUA+. For attribute weighting, a qualitative analysis pre-step using rough set analysis (RSA) is performed. RSA is a proven machine learning technique for classification of objects. We exploit the RSA results in different ways and define four heuristics for attribute weighting. AQUA+ was evaluated in two ways: (1) comparison between AQUA+ and AQUA, along with the comparative analysis between the proposed four heuristics for AQUA+, (2) comparison of AQUA+ with other EBA methods. The main evaluation results are: (1) better estimation accuracy was obtained by AQUA+ compared to AQUA over all six data sets; and (2) AQUA+ obtained better results than, or very close to that of other EBA methods for the three data sets applied to all the EBA methods. In conclusion, the proposed attribute weighing method using RSA can improve the estimation accuracy of EBA method AQUA+ according to the empirical studies over six data sets. Testing more data sets is necessary to get results that are more statistical significant.

COTS Selection: Past, Present, and Future

Commercial off-the-shelf (COTS) products are increasingly being used in software development. In COTS-based development, selecting appropriate COTS is the most crucial phase. This paper explores the evolution of COTS selection practices, and surveys eighteen of the most significant COTS selection approaches. The paper traces how each approach contributed to the improvement of current COTS selection practices, and then compares them .The paper also highlights some open research issues relevant to the selection process, and concludes with a discussion of possible future directions to address these issues

Software release planning for evolving systems

Release planning is a crucial step in incremental software development. It addresses the issues involved with assigning features to sequence of releases of a system such that the most important technical, resource, risk and budget constraints are met. These problems are difficult to solve for even mid-sized systems. The issues become even more challenging in evolving systems where we need to consider the characteristics of the existing system, as the existing components of the system have their own history and status in terms of size, complexity, health, criticality, and understandability.In this paper, we present the foundations for handling release planning for evolving systems in a rigorous manner. Based on a formalized problem description, we present a new solution approach for release planning of evolving systems called S-EVOLVE*. From analyzing and comparing different characteristics of the target components, where new features will be implemented, we obtain a more detailed perspective of the potential impact of implementing one feature or another. As part of this analysis, we have applied the analytic hierarchy process (AHP) to define weighting factors for component modifiability. The information gained is used for designing release plans based on thresholds for the relative extent of modifiability acceptable for a release. A set of structurally different release plans is generated based on solving a specialized integer linear-programming problem. The plans are proven to be semi-optimal for the stated objectives.A case study is performed to demonstrate the added value of the approach. The evolving system under consideration is the intelligent decision-support tool ReleasePlanner. We compare and discuss results, for planning future releases, for the cases with and without consideration of system constraints.

Optimized assignment of developers for fixing bugs an initial evaluation for eclipse projects

Decisions on “Who should fix this bug” have substantial impact on the duration of the process and its results. In this paper, optimized strategies for the assignment of the “right” developers for doing the “right” task are studied and the results are compared to manual (called ad hoc) assignment. The quality of assignment is measured by the match between requested (from bugs) and available (from developers) competence profile. Different variants of Greedy search with varying parameter of look-ahead time are studied. The quality of the results has been evaluated for nine milestones of the open source Eclipse JDT project. The optimized strategies with largest look ahead time are demonstrated to be substantially better than the ad hoc solutions in terms of the quality of the assignment and the number of bugs which can be fixed within the given time interval.