Masateru Tsunoda

Effort Estimation Based on Collaborative Filtering

Effort estimation methods are one of the important tools for project managers in controlling human resources of ongoing or future software projects. The estimations require historical project data including process and product metrics that characterize past projects. Practically, in using the estimation methods, it is a problem that the historical project data frequently contain substantial missing values. In this paper, we propose an effort estimation method based on Collaborative Filtering for solving the problem. Collaborative Filtering has been developed in information retrieval researchers, as one of the estimation techniques using defective data, i.e. data having substantial missing values. The proposed method first evaluates similarity between a target (ongoing) project and each past project, using vector based similarity computation equation. Then it predicts the effort of the target project with the weighted sum of the efforts of past similar projects. We conducted an experimental case study to evaluate the estimation performance of the proposed method. The proposed method showed better performance than the conventional regression method when the data had substantial missing values.

Analyzing OSS developers' working time using mailing lists archives

2. INPUT DATA We used mailing lists (MLs) archives of PostgreSQL, downloaded from http://www.postgresql.org/community/lists/. The MLs mainly consist of user lists and developer lists. We used developer lists archive since we needed developers’ working time. Table 1 explains details of each ML. Figure 1 shows amounts of messages of each ML in the developer lists. Amounts of messages were increasing year by year. The ML of hackers had many more messages than other MLs. We extracted MLs archives till December 2005. Note that most of committers’ messages were automatically generated when source code was checked into software configuration management repository. We picked up “mail sent time” to identify developers’ working time. Getting mail sent time from the MLs archives consists of the following two steps: First, we downloaded the MLs archives with 0 500

Software development productivity of Japanese enterprise applications

To clarify the relationship between software development productivity and the attributes of a software project, such as business area, programming language and team size, this paper analyzed 211 enterprise application development projects in Japan using a software engineering data repository established by the Software Engineering Center (SEC), Information-Technology Promotion Agency, Japan. In the analysis, we first identified factors that related to productivity based on a parallel coordinate plot (PCP) and a one-way ANOVA. An in-depth analysis on each productivity factor was then conducted by selecting a project subset for each factor so that the effect of other factors is minimized. Our findings include that the average team size was the strongest attribute relating to productivity. The outsourcing ratio (percentage), which can be controlled by software development companies, and the business sector both showed a moderate relationship to productivity. Finally, product size (FP), the duration of development and the programming language were only weakly related to productivity.

Productivity analysis of Japanese enterprise software development projects

To clarify the relation between controllable attributes of a software development and its productivity, this paper experimentally analyzed a software project repository (SEC repository), consisting of 253 enterprise software development projects in Japanese companies, established by Software Engineering Center (SEC), Information-technology Promotion Agency, Japan. In the experiment, as controllable attributes, we focused on the outsourcing ratio of a software project, defined as an effort outsourced to subcontract companies divided by a whole development effort, and on the effort allocation balance among development phases. Our major findings include both larger outsourcing ratio and smaller upstream process effort leads to worse productivity.

An empirical evaluation of outlier deletion methods for analogy-based cost estimation

Background: Any software project dataset sometimes includes outliers which affect the accuracy of effort estimation. Outlier deletion methods are often used to eliminate them. However, there are few case studies which apply outlier deletion methods to analogy-based estimation, so it is not clear which method is more suitable for analogy-based estimation. Aim: Clarifying the effects of existing outlier deletion methods (Cooks distance based deletion, LTS based deletion, k-means based deletion, Mantels correlation based deletion, and EID based deletion) and our method for analogy-based estimation. Method: In the experiment, outlier deletion methods were applied to three kinds of datasets (the ISBSG, Kitchenham, and Desharnais datasets), and their estimation accuracy evaluated based on BRE (Balanced Relative Error). Our method eliminates outliers from the neighborhoods of a target project when the effort is extremely different from other neighborhoods. Results: Deletion methods which are designed to apply to analogy-based estimation (i.e. Mantels correlation based deletion, EID based deletion, and our method) showed stable performance. Especially, only our method showed over 10% improvement of the average BRE on two datasets. Conclusions: It is reasonable to apply deletion methods designed for analogy-based estimation, and more preferable to apply our method to analogy-based estimation.

Benchmarking Software Maintenance Based on Working Time

Software maintenance is an important activity on the software lifecycle. Software maintenance does not mean only removing faults found after software release. Software needs extensions or modifications of its functions due to changes in a business environment, and software maintenance also indicates them. In this research, we try to establish a benchmark of work efficiency for software maintenance. To establish the benchmark, factors affecting work efficiency should be clarified, using a dataset collected from various organizations (cross-company dataset). We used dataset includes 134 data points collected by Economic Research Association in 2012, and analyzed factors affected work efficiency of software maintenance. We defined the work efficiency as number of modified modules divided by working time. The main contribution of our research is illustrating factors affecting work efficiency, based on the analysis using cross-company dataset and working time. Also, we showed work efficiency, classified the factor. It can be used to benchmark an organization. We empirically illustrated that using Java and restriction of development tool affect to work efficiency.

Revisiting software development effort estimation based on early phase development activities

Many research projects on software estimation use software size as a major explanatory variable. However, practitioners sometimes use the ratio of effort for early phase activities such as planning and requirement analysis, to the effort for the whole development phase of the software in order to estimate effort. In this paper, we focus on effort estimation based on the effort for early phase activities. The goal of the research is to examine the relationship of early phase effort and software size with software development effort. To achieve the goal, we built effort estimation models using early phase effort as an explanatory variable, and compared the estimation accuracies of these models to the effort estimation models based on software size. In addition, we built estimation models using both early phase effort and software size. In our experiment, we used ISBSG dataset, which was collected from software development companies, and regarded planning phase effort and requirement analysis effort as early phase effort. The result of the experiment showed that when both software size and sum of planning and requirement analysis phase effort were used as explanatory variables, the estimation accuracy was most improved (Average Balanced Relative Error was improved to 75.4% from 148.4%). Based on the result, we recommend that both early phase effort and software size be used as explanatory variables, because that combination showed the high accuracy, and did not have multicollinearity issues.

Modeling Software Project Monitoring with Stakeholders

Recently, software size becomes larger, and consequently, not only a software developer but also a software purchaser suffers considerable losses by software project failure. So avoiding project failure is also important for purchasers. Project monitoring with a purchaser and a developer (stakeholders) is expected for the purchaser to suppress risk of project failure. It is performed by sharing software metrics during the project for the purchaser to grasp the status of the project. Although there are some software measurement models, they cannot describe two kinds of metrics which are used to monitor projects with stakeholders. One metric is to indicate project goal achievement after finishing project. The other one is to measure to progress toward the goal. In addition, the models cannot represent countermeasures when symptoms of project failure are found. We propose the model for project monitoring with stakeholders. The model is based on the measurement information model defined by ISO/IEC 15939, and added stakeholder’s goal, key goal indicator (KGI), key performance indicator (KPI), corrective action, and check timing. With our model, project monitoring with stakeholders can be described more rigorously.

Recommendation of software technologies based on collaborative filtering

Software engineers have to select some appropriate development technologies to use in the work; however, engineers sometimes cannot find the appropriate technologies because there are vast amount of options today. To solve this problem, we propose a software technology recommendation method based on collaborative filtering (CF). In the proposed method, at first, questionnaires are collected from concerned engineers about their technical interest. Next, similarities between an active engineer who gets recommendation and the other engineers are calculated according to the technical interests. Then, some similar engineers are selected for the active engineer. At last, some technologies are recommended which attract the similar engineers. An experimental evaluation showed that the proposed method can make accurate recommendations than that of a naive (non-CF) method.

Pitfalls of analyzing a cross-company dataset of software maintenance and support

It is important to establish a benchmark of work efficiency for software maintenance and support. For maintenance and support service providers, the benchmarking is the basis for improvement of their work. For the service purchasers, it is useful to check work efficiency of contracted service provider. To establish a benchmark of work efficiency for software development activities, a cross-company dataset is often used. Data points included in it are collected from various organizations. ISBSG (International Software Benchmarking Standards Group) builds the cross-company dataset of software maintenance and support. In the analysis, we found some pitfalls when one analyzes ISBSG software maintenance and support dataset. If the pitfalls are ignored, spurious relationships would be found. In this paper, we showed some pitfalls, and how to avoid it. It would be very useful for researchers, because ISBSG software maintenance and support dataset may be widely used in the near future.