Marta Fernández-Diego

Potential and limitations of the ISBSG dataset in enhancing software engineering research: A mapping review

Context: The International Software Benchmarking Standards Group (ISBSG) maintains a software development repository with over 6000 software projects. This dataset makes it possible to estimate a projects size, effort, duration, and cost. Objective: The aim of this study was to determine how and to what extent, ISBSG has been used by researchers from 2000, when the first papers were published, until June of 2012. Method: A systematic mapping review was used as the research method, which was applied to over 129 papers obtained after the filtering process. Results: The papers were published in 19 journals and 40 conferences. Thirty-five percent of the papers published between years 2000 and 2011 have received at least one citation in journals and only five papers have received six or more citations. Effort variable is the focus of 70.5% of the papers, 22.5% center their research in a variable different from effort and 7% do not consider any target variable. Additionally, in as many as 70.5% of papers, effort estimation is the research topic, followed by dataset properties (36.4%). The more frequent methods are Regression (61.2%), Machine Learning (35.7%), and Estimation by Analogy (22.5%). ISBSG is used as the only support in 55% of the papers while the remaining papers use complementary datasets. The ISBSG release 10 is used most frequently with 32 references. Finally, some benefits and drawbacks of the usage of ISBSG have been highlighted. Conclusion: This work presents a snapshot of the existing usage of ISBSG in software development research. ISBSG offers a wealth of information regarding practices from a wide range of organizations, applications, and development types, which constitutes its main potential. However, a data preparation process is required before any analysis. Lastly, the potential of ISBSG to develop new research is also outlined.

Estimating future bidding performance of competitor bidders in capped tenders

Research in Bid Tender Forecasting Models (BTFM) has been in progress since the 1950s. None of the developed models were easy-to-use tools for effective use by bidding practitioners because the advanced mathematical apparatus and massive data inputs required. This scenario began to change in 2012 with the development of the Smartbid BTFM, a quite simple model that presents a series of graphs that enables any project manager to study competitors using a relatively short historical tender dataset. However, despite the advantages of this new model, so far, it is still necessary to study all the auction participants as an indivisible group; that is, the original BTFM was not devised for analyzing the behavior of a single bidding competitor or a subgroup of them. The present paper tries to solve that flaw and presents a stand-alone methodology useful for estimating future competitors’ bidding behaviors separately.

The usage of ISBSG data fields in software effort estimation

We performed a systematic mapping study over 107 papers that use ISBSG data for effort estimation.Usage of dependent, filtering, and independent variables in effort models is described.Factors that guide the selection of relevant independent variables are described.Influence of estimation methods in the selection of variables has been outlined.References that have worked with a specific independent variable are listed. The International Software Benchmarking Standards Group (ISBSG) maintains a repository of data about completed software projects. A common use of the ISBSG dataset is to investigate models to estimate a software projects size, effort, duration, and cost. The aim of this paper is to determine which and to what extent variables in the ISBSG dataset have been used in software engineering to build effort estimation models. For that purpose a systematic mapping study was applied to 107 research papers, obtained after a filtering process, that were published from 2000 until the end of 2013, and which listed the independent variables used in the effort estimation models. The usage of ISBSG variables for filtering, as dependent variables, and as independent variables is described. The 20 variables (out of 71) mostly used as independent variables for effort estimation are identified and analysed in detail, with reference to the papers and types of estimation methods that used them. We propose guidelines that can help researchers make informed decisions about which ISBSG variables to select for their effort estimation models.

How the Weather Impacts on the Performance of an Outdoor WLAN

It is believed that meteorological factors such as rain, wind or temperature do not affect significantly to wireless systems operating at radio frequencies below 10 GHz. However, using an appropriate statistical analysis to the measurements taken from an outdoor IEEE 802.11b/g WLAN, we show that this is not true. Although these networks operate at 2.4 GHz, we have been able to prove a significant relationship between some meteorological factors and the behavior of several control frames in these networks.

Sensitivity of results to different data quality meta-data criteria in the sample selection of projects from the ISBSG dataset

Background: Most prediction models, e.g. effort estimation, require preprocessing of data. Some datasets, such as ISBSG, contain data quality meta-data which can be used to filter out low quality cases from the analysis. However, an agreement has not been reached yet between researchers about these data quality selection criteria. Aims: This paper aims to analyze the influence of data quality meta-data criteria in the number of selected projects, which can have influence in the models obtained. For this, a case study has been selected to gain a more complete understanding of what might be important to focus in future research. Method: Data quality meta-data selection criteria of some works based on ISBSG dataset which propose prediction models were reviewed first. Considerable attention has been paid to two data quality meta-data variables in ISBSG dataset Release 11 which are Data Quality Rating and Unadjusted Function Point Rating. Secondly, this paper considers data from 830 projects which have been collected from the ISBSG dataset after a preliminary screening. This first screening leads mainly to a subset of projects with comparable definitions in size and effort. Then data quality meta-data criteria are applied in order to infer their influence. Results: Overall, it seems that data selection criteria, regardless data quality meta-data concerns, involve an important reduction in sample size. From 5052 projects, only 830 are really considered. Then 262 projects remain for analysis if the maximum quality rate is applied for both data quality meta-data variables. But, since the initial data preparation focuses the problem of missingness for a certain purpose, data quality criteria seem not to be the clue for the analysis results. However, some variability has been observed. Conclusions: Whilst this analysis is supported by a case study, it is hoped that it contributes to a better understanding of the subject. In fact, results found suggest that in those studies where the selection criteria of projects are not very strictly applied, these data quality criteria must be carefully taken into account.

ISBSG variables most frequently used for software effort estimation: a mapping review

Background: The International Software Benchmarking Standards Group (ISBSG) dataset makes it possible to estimate a projects size, effort, duration, and cost. Aim: The aim was to analyze the ISBSG variables that have been used by researchers for software effort estimation from 2000, when the first papers were published, until the end of 2013. Method: A systematic mapping review was applied to over 167 papers obtained after the filtering process. From these, it was found that 133 papers produce effort estimation and only 107 list the independent variables used in the effort estimation models. Results: Seventy-one out of 118 ISBSG variables have been used at least once. There is a group of 20 variables that appear in more than 50% of the papers and include Functional Size (62%), Development Type (58%), Language Type (53%), and Development Platform (52%) following ISBSG recommendations. Sizing and Size attributes altogether represent the most relevant group along with Project attributes that includes 24 technical features of the project and the development platform. All in all, variables that have more missing values are used less frequently. Conclusions: This work presents a snapshot of the existing usage of ISBSG variables in software development estimation. Moreover, some insights are provided to guide future studies.

Discretization methods for NBC in effort estimation: an empirical comparison based on ISBSG projects

Background: Bayesian networks have been applied in many fields, including effort estimation in software engineering. Even though there are Bayesian inference algorithms than can handle continuous variables, performance tends to be better when these variables are discretized that when they are assumed to follow a specific distribution. On the other hand, the choice of the discretization method and the number of discretized intervals may lead to significantly different estimating results. However, discretization issues are seldom mentioned in software engineering effort estimation models. Aim: This paper seeks to show that discretization issues are important in terms of prediction accuracy while building a Naive Bayes Classifier (NBC) for estimating software effort. Method: For this purpose, a NBC model has been developed for software effort estimation based on ISBSG projects applying different discretization schemes (equal width intervals, equal frequency intervals, and k-means clustering) and using different number of intervals. Results: Regarding the NBC model built, the estimation accuracy of equal frequency discretization is only improved by k-means clustering with respect to Pred(0.25), although it reflects better the original distribution. Conclusions: Further experimentation should determine the potential of clustering methods already highlighted in other fields.

Measuring Competencies in Higher Education. The Case of Innovation Competence

Within the context of permanent change, innovation has become a vital value for the survival and development of the organisations. The development of this increasingly important value will help students to gain access to the labour market and to adapt to their future jobs in accordance with these characteristics. Competency describes what training participants should be able to do at the end of such training. Competency is acquired through the various learning objectives to be achieved. Innovation competency is closely related to Self-assessment and the learning methods, Ability to work in interactive communication situations, Ability to create and maintain connections work, Ability to cooperate in a multidisciplinary and multicultural environment and Ability to communicate and interact in an international environment, etc. In this chapter, we develop a method for measuring the innovation competencies in higher education by introducing different levels of mastery.

Software Effort Estimation Using NBC and SWR: A Comparison Based on ISBSG Projects

There are many quantitative estimation methods, e.g. linear regression, neural networks, regression trees. Compared to traditional methods, Bayesian networks are being increasingly used in software engineering because their use opens many possibilities. A main feature of Bayesian networks is their capability to combine data and expert knowledge. This paper seeks to reinforce the hypothesis that Bayesian networks are a competitive method for estimating software effort in terms of prediction accuracy. For this purpose a Naive Bayes Classifier (NBC) and a forward Stepwise Regression (SWR) models have been developed from a subset of the ISBSG dataset. Under homogeneous conditions we found similar results provided that the discretization of the continuous variables is thin enough.

‘Driving’ IS projects

Using a didactical car-driving metaphor, this paper deals with a proactive way of ‘driving’ projects with high uncertainty. The centrality of problem complexity and problem uncertainty are demonstrated, the mapping of these to human cognition is reviewed, and the car-driving metaphor as ‘driving’ project management is developed. The MadPRYX ‘suite’ develops didactic car-driving models and practical Scoreboard metrics, looking for successful equilibrium of effectiveness-efficiency, conditioned by the levels and types of complexity and uncertainty in the project process and its environment