Gunnar R. Bergersen

Inferring Skill from Tests of Programming Performance: Combining Time and Quality

The skills of software developers are important to the success of software projects. Also, when studying the general effect of a tool or method, it is important to control for individual differences in skill. However, the way skill is assessed is often ad hoc, or based on unvalidated methods. According to established test theory, validated tests of skill should infer skill levels from well-defined performance measures on multiple, small, representative tasks. In this respect, we show how time and quality, which are often analyzed separately, can be combined as task performance and subsequently be aggregated as an approximation of skill. Our results show significant positive correlations between our proposed measures of skill and other variables, such as seniority, lines of code written, and self-evaluated expertise. The method for combining time and quality is a promising first step to measuring programming skill in both industry and research settings.

Teamwork quality and project success in software development

We studied the effect of teamwork quality on project success in agile software teams.We ran a survey with responses from 477 respondents from 71 teams in 26 companies.Teamwork quality is perceived to have a small to large effect on team performance, depending of the rater.Teamwork quality is perceived to have a large effect on personal success.Teamwork quality and its effects are not greater in agile than in traditional teams. Small, self-directed teams are central in agile development. This article investigates the effect of teamwork quality on team performance, learning and work satisfaction in agile software teams, and whether this effect differs from that of traditional software teams. A survey was administered to 477 respondents from 71 agile software teams in 26 companies and analyzed using structural equation modeling. A positive effect of teamwork quality on team performance was found when team members and team leaders rated team performance. In contrast, a negligible effect was found when product owners rated team performance. The effect of teamwork quality on team membersź learning and work satisfaction was strongly positive, but was only rated by the team members. Despite claims of the importance of teamwork in agile teams, this study did not find teamwork quality to be higher than in a similar survey on traditional teams. The effect of teamwork quality on team performance was only marginally greater for the agile teams than for the traditional teams.

Evaluating methods and technologies in software engineering with Respect to Developers' skill level

Background: It is trivial that the usefulness of a technology depends on the skill of the user. Several studies have reported an interaction between skill levels and different technologies, but the effect of skill is, for the most part, ignored in empirical, human-centric studies in software engineering. Aim: This paper investigates the usefulness of a technology as a function of skill. Method: An experiment that used students as subjects found recursive implementations to be easier to debug correctly than iterative implementations. We replicated the experiment by hiring 65 professional developers from nine companies in eight countries. In addition to the debugging tasks, performance on 17 other programming tasks was collected and analyzed using a measurement model that expressed the effect of treatment as a function of skill. Results: The hypotheses of the original study were confirmed only for the low-skilled subjects in our replication. Conversely, the high-skilled subjects correctly debugged the iterative implementations faster than the recursive ones, while the difference between correct and incorrect solutions for both treatments was negligible. We also found that the effect of skill (odds ratio = 9.4) was much larger than the effect of the treatment (odds ratio = 1.5). Conclusions: Claiming that a technology is better than another is problematic without taking skill levels into account. Better ways to assess skills as an integral part of technology evaluation are required.

Teamwork Quality and Team Performance: Exploring Differences Between Small and Large Agile Projects

Agile principles were originally developed for small projects but are now widely used in larger projects with hundreds of developers. Teamwork quality is essential in any development work, but how does teamwork quality differ in small and large agile projects? We report from an explorative survey with 64 agile teams and 320 team members and team leaders, from 31 teams in small projects and 33 teams in large projects. For small projects, teamwork quality was considered by both team members and team leaders to primarily affect product quality. For large projects, the effect of teamwork quality on product quality was positive when it was rated by team members but was negative when rated by team leaders. At a finer granularity, the six dimensions of teamwork quality that we investigated affected team performance differently in small and large projects. These findings question to what extent findings from previous studies on teamwork in agile development in small projects apply to large projects.

Why theory matters

Abstract It is relatively easy to generate and acquire much data from software engineering activities. The challenge is to obtain meaning from the data that represents something true, rather than spurious. To increase knowledge and insight, more theories should be built and used.

Evidence-based software engineering

A decade ago, Kitchenham, Dyba and Jorgensen coined the term and provided the foundations for evidence-based software engineering (EBSE). A trilogy of papers was written for researchers, practitioners, and educators. They suggested that practitioners consider EBSE as a mechanism to support and improve their technology adoption decisions, and that researchers should use systematic literature reviews as a methodology for performing unbiased aggregation of empirical results. This spurred significant international activity, and a renewed focus on research methods and theory, and on the future of empirical methods in SE research.