Chris Lokan

Neural-Based Learning Classifier Systems

UCS is a supervised learning classifier system that was introduced in 2003 for classification in data mining tasks. The representation of a rule in UCS as a univariate classification rule is straightforward for a human to understand. However, the system may require a large number of rules to cover the input space. Artificial neural networks (NNs), on the other hand, normally provide a more compact representation. However, it is not a straightforward task to understand the network. In this paper, we propose a novel way to incorporate NNs into UCS. The approach offers a good compromise between compactness, expressiveness, and accuracy. By using a simple artificial NN as the classifiers action, we obtain a more compact population size, better generalization, and the same or better accuracy while maintaining a reasonable level of expressiveness. We also apply negative correlation learning (NCL) during the training of the resultant NN ensemble. NCL is shown to improve the generalization of the ensemble.

Software project effort estimation using genetic programming

Knowing the estimated cost of a software project early in the development cycle is a valuable asset for management. In this paper, an evolutionary computation method, grammar guided genetic programming (GGGP), is used to fit models, with the aim of improving the prediction of software development costs. Valuable results are obtained, significantly better than those obtained by simple linear regression. In this research, GGGP, because of its flexibility and the ability of incorporating background knowledge, also shows great potential in being applied in other software engineering modeling problems.

Applying moving windows to software effort estimation

Models for estimating software development effort are commonly built and evaluated using a set of historical projects. An important question is which projects to use as training data to build the model: should it be all of them, or a subset that seems particularly relevant? One factor to consider is project age: is it best to use the entire history of past projects, or is it more appropriate in a rapidly changing world to use a window of recent projects? We investigate the effect on estimation accuracy of using a moving window, using projects from the ISBSG data set. We find that using a moving window can improve accuracy, and we make some observations about factors that influence the range of possible window sizes and the best window size.

An empirical analysis of function point adjustment factors

Abstract In function point analysis, fourteen “general systems characteristics” (GSCs) are used to construct a “value adjustment factor” (VAF), with which a basic function point count is adjusted. Although the GSCs and VAF have been criticized on both theoretical and practical grounds, they are used by many practitioners. This paper reports on an empirical investigation into their use and practical value. We conclude that recording the GSCs may be useful for understanding project cost drivers and for comparing similar projects, but the VAF should not be used: doubts about its construction are not balanced by any practical benefit. A new formulation is needed for using the GSCs to explain effort; factors identified here could guide further research.

A Function Point-Like Measure for Object-Oriented Software

We present a method for estimating the size, and consequently effort and duration, of object oriented software development projects. Different estimates may be made in different phases of the development process, according to the available information. We define an adaptation of traditional function points, called “Object Oriented Function Points”, to enable the measurement of object oriented analysis and design specifications. Tools have been constructed to automate the counting method. The novel aspect of our method is its flexibility. An organization can experiment with different counting policies, to find the most accurate predictors of size, effort, etc. in its environment. The method and preliminary results of its application in an industrial environment are presented and discussed.

A replicated comparison of cross-company and within-company effort estimation models using the ISBSG database

Four years ago was the last time the ISBSG database was used to compare the effort prediction accuracy between cross-company and within-company cost models. Since then more than 2,000 projects have been volunteered to this database, which may have changed the trends previously observed. This paper therefore replicates a previous study by investigating how successful a cross-company cost model is: i) to estimate effort for projects that belong to a single company and were not used to build the cross-company model; ii) compared to a within-company cost model. Our within-company data set had data on 184 software projects from a single company and our cross-company data set employed data on 672 software projects. Our results did not corroborate those from the previous study, showing that predictions based on the within-company model were not significantly more accurate than those based on the cross-company model. We analysed the data using forward stepwise regression

Cross-company and single-company effort models using the ISBSG database: a further replicated study

Five years ago the ISBSG database was used by Jeffery et al. [6] (S1) to compare the effort prediction accuracy between cross- and single-company effort models. Given that more than 2,000 projects were later volunteered to this database, in 2005 Mendes et al. [17] (S2) replicated S1 but obtained different results. The difference in results between both studies could have resulted from legitimate differences in data set patterns but also could have been influenced by differences in experimental procedure. S2 was unable to employ exactly the same experimental procedure used in S1, as S1s procedure was not fully documented. Therefore this paper aimed to apply S2s experimental procedure to the ISBSG database version used in S1 (release 6) to assess if differences in experimental procedure would have contributed towards different results. Our results corroborated those from S1: we found that predictions based on a single-company model were significantly more accurate than those based on a cross-company model.

Replicating studies on cross- vs single-company effort models using the ISBSG Database

In 2001 the ISBSG database was used by Jeffery et al. (Using public domain metrics to estimate software development effort. Proceedings Metrics’01, London, pp 16–27, 2001; S1) to compare the effort prediction accuracy between cross- and single-company effort models. Given that more than 2,000 projects were later volunteered to this database, in 2005 Mendes et al. (A replicated comparison of cross-company and within-company effort estimation models using the ISBSG Database, in Proceedings of Metrics’05, Como, 2005; S2) replicated S1 but obtained different results. The difference in results could have occurred due to legitimate differences in data set patterns; however, they could also have occurred due to differences in experimental procedure given that S2 was unable to employ exactly the same experimental procedure used in S1 because S1’s procedure was not fully documented. Recently, we applied S2’s experimental procedure to the ISBSG database version used in S1 (release 6) to assess if differences in experimental procedure would have contributed towards different results (Lokan and Mendes, Cross-company and single-company effort models using the ISBSG Database: a further replicated study, Proceedings of the ISESE’06, pp 75–84, 2006; S3). Our results corroborated those from S1, suggesting that differences in the results obtained by S2 were likely caused by legitimate differences in data set patterns. We have since been able to reconstruct the experimental procedure of S1 and therefore in this paper we present both S3 and also another study (S4), which applied the experimental procedure of S1 to the data set used in S2. By applying the experimental procedure of S2 to the data set used in S1 (study S3), and the experimental procedure of S1 to the data set used in S2 (study S4), we investigate the effect of all the variations between S1 and S2. Our results for S4 support those of S3, suggesting that differences in data preparation and analysis procedures did not affect the outcome of the analysis. Thus, the different results of S1 and S2 are very likely due to fundamental differences in the data sets.

Definition and experimental evaluation of function points for object-oriented systems

We present a method for estimating the size, and consequently effort and duration, of object oriented software development projects. Different estimates may be made in different phases of the development process, according to the available information. We define an adaptation of traditional function points, called Object Oriented Function Points, to enable the measurement of object oriented analysis and design specifications. Tools have been constructed to automate the counting method. The novel aspect of our method is its flexibility. An organisation can experiment with different counting policies, to find the most accurate predictors of size, effort, etc. in its environment. The method and preliminary results of its application in an industrial environment are presented and discussed.

AMA: a new approach for solving constrained real-valued optimization problems

Memetic algorithms (MA) have recently been applied successfully to solve decision and optimization problems. However, selecting a suitable local search technique remains a critical issue of MA, as this significantly affects the performance of the algorithms. This paper presents a new agent based memetic algorithm (AMA) for solving constrained real-valued optimization problems, where the agents have the ability to independently select a suitable local search technique (LST) from our designed set. Each agent represents a candidate solution of the optimization problem and tries to improve its solution through co-operation with other agents. Evolutionary operators consist of only crossover and one of the self-adaptively selected LSTs. The performance of the proposed algorithm is tested on five new benchmark problems along with 13 existing well-known problems, and the experimental results show convincing performance.