Hakan Erdogmus

Operational definition and automated inference of test-driven development with Zorro

Test-driven development (TDD) is a style of development named for its most visible characteristic: the design and implementation of test cases prior to the implementation of the code required to make them pass. Many claims have been made for TDD: that it can improve implementation as well as design quality, that it can improve productivity, that it results in 100% coverage, and so forth. However, research to validate these claims has yielded mixed and sometimes contradictory results. We believe that at least part of the reason for these results stems from differing interpretations of the TDD development style, along with an inability to determine whether programmers actually follow whatever definition of TDD is in use.Zorro is a system designed to automatically determine whether a developer is complying with an operational definition of Test-Driven Development (TDD) practices. Automated recognition of TDD can benefit the software development community in a variety of ways, from inquiry into the “true nature” of TDD, to pedagogical aids to support the practice of test-driven development, to support for more rigorous empirical studies on the effectiveness of TDD in both laboratory and real world settings.This paper describes the Zorro system, its operational definition of TDD, the analyses made possible by Zorro, two empirical evaluations of the system, and an attempted case study. Our research shows that it is possible to define an operational definition of TDD that is amenable to automated recognition, and illustrates the architectural and design issues that must be addressed in order to do so. Zorro has implications not only for the practice of TDD, but also for software engineering “micro-process” definition and recognition through its parent framework, Software Development Stream Analysis.

Topic selection in industry experiments

This paper shares our experience with initial negotiation and topic elicitation process for conducting industry experiments in six software development organizations in Finland. The process involved interaction with company representatives in the form of both multiple group discussions and separate face-to-face meetings. Fitness criteria developed by researchers were applied to the list of generated topics to decide on a common topic. The challenges we faced include diversity of proposed topics, communication gaps, skepticism about research methods, initial disconnect between research and industry needs, and lack of prior work relationship. Lessons learned include having enough time to establish trust with partners, importance of leveraging the benefits of training and skill development that are inherent in the experimental approach, uniquely positioning the experimental approach within the landscape of other validation approaches more familiar to industrial partners, and introducing the fitness criteria early in the process.

A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?

Background: Test-driven development (TDD) is a technique that repeats short coding cycles interleaved with testing. The developer first writes a unit test for the desired functionality, followed by the necessary production code, and refactors the code. Many empirical studies neglect unique process characteristics related to TDD iterative nature. Aim: We formulate four process characteristic: sequencing, granularity, uniformity, and refactoring effort. We investigate how these characteristics impact quality and productivity in TDD and related variations. Method: We analyzed 82 data points collected from 39 professionals, each capturing the process used while performing a specific development task. We built regression models to assess the impact of process characteristics on quality and productivity. Quality was measured by functional correctness. Result: Quality and productivity improvements were primarily positively associated with the granularity and uniformity. Sequencing, the order in which test and production code are written, had no important influence. Refactoring effort was negatively associated with both outcomes. We explain the unexpected negative correlation with quality by possible prevalence of mixed refactoring. Conclusion: The claimed benefits of TDD may not be due to its distinctive test-first dynamic, but rather due to the fact that TDD-like processes encourage fine-grained, steady steps that improve focus and flow.

CodeAware: sensor-based fine-grained monitoring and management of software artifacts

Current continuous integration (CI) tools, although extensible, can be limiting in terms of flexibility. In particular, artifact analysis capabilities available through plug in mechanisms are both coarse-grained and centralized. To address this limitation, this paper introduces a new paradigm, Code Aware, for distributed and fine-grained artifact analysis. Code Aware is an ecosystem inspired by sensor networks, consisting of monitors and actuators, aimed at improving code quality and team productivity. Code wares vision entails (a) the ability to probe software artifacts of any granularity and localization, from variables to classes or files to entire systems, (b) the ability to perform both static and dynamic analyses on these artifacts, and (c) the ability to describe targeted remediation actions, for example to notify interested developers, through automated actuators. We provide motivational examples for the use of Code Aware that leverage current CI solutions, sketch the architecture of its underlying ecosystem, and outline research challenges.

An industry experiment on the effects of test-driven development on external quality and productivity

Existing empirical studies on test-driven development (TDD) report different conclusions about its effects on quality and productivity. Very few of those studies are experiments conducted with software professionals in industry. We aim to analyse the effects of TDD on the external quality of the work done and the productivity of developers in an industrial setting. We conducted an experiment with 24 professionals from three different sites of a software organization. We chose a repeated-measures design, and asked subjects to implement TDD and incremental test last development (ITLD) in two simple tasks and a realistic application close to real-life complexity. To analyse our findings, we applied a repeated-measures general linear model procedure and a linear mixed effects procedure. We did not observe a statistical difference between the quality of the work done by subjects in both treatments. We observed that the subjects are more productive when they implement TDD on a simple task compared to ITLD, but the productivity drops significantly when applying TDD to a complex brownfield task. So, the task complexity significantly obscured the effect of TDD. Further evidence is necessary to conclude whether TDD is better or worse than ITLD in terms of external quality and productivity in an industrial setting. We found that experimental factors such as selection of tasks could dominate the findings in TDD studies.

Flipping a graduate-level software engineering foundations course

Creating a graduate-level software engineering breadth course is challenging. The scope is wide. Students prefer hands-on work over theory. Industry increasingly values soft skills. Changing software technology requires the syllabus to be technology-agnostic, yet abstracting away technology compromises realism. Instructors must balance scope with depth of learning. At Carnegie Mellon University, we designed a flipped-classroom course that tackles these tradeoffs. The course has been offered since Fall 2014 in the Silicon Valley campus. In this paper, we describe the courses key features and summarize our experiences and lessons learned while designing, teaching, and maintaining it. We found that the pure flipped-classroom format was not optimal in ensuring sufficient transfer of knowledge, especially in remote settings. We initially underestimated teaching assistantship resources. We gradually complemented video lectures and hands-on live sessions with additional live components: easily replaceable recitations that focus on current technology and mini lectures that address application of theory and common wisdom. We also provided the students with more opportunities to share their successes and experiments with their peers. We achieved scalability by increasing the number of teaching assistants, paying attention to teaching assistant recruitment, and fostering a culture of mentoring among the teaching team.

Small Polygon Compression

We introduce a compression technique for small polygons. The main application is to embed compressed polygons in emergency alert messages that have strict length restrictions, as in the case of Wireless Emergency Alert messages. We transform polygon coordinates to sets of integers and are able to compress them to between 10.4% and 25.6% of original length reducing original polygon lengths from 43-331 characters to 9-61 characters. The compression technique introduced in this work takes advantage of a strongly skewed polygon coordinate distribution.

Lightweight source code monitoring with Triggr

Existing tools for monitoring the quality of codebases modified by multiple developers tend to be centralized and inflexible. These tools increase the visibility of quality by producing effective reports and visualizations when a change is made to the codebase and triggering alerts when undesirable situations occur. However, their configuration is invariably both (a) centrally managed in that individual maintainers cannot define local rules to receive customized feedback when a change occurs in a specific part of the code in which they are particularly interested, and (b) coarse-grained in that analyses cannot be turned on and off below the file level. Triggr, the tool proposed in this paper, addresses these limitations by allowing distributed, customized, and fine-grained monitoring. It is a lightweight re-implementation of our previous tool, CodeAware, which adopts the same paradigm. The tool listens on a codebases shared repository using an event-based approach, and can send alerts to subscribed developers based on rules defined locally by them. Triggr is open-source and available at https://github.com/lyzerk/Triggr. A demonstration video can be found at https://youtu.be/qQs9aDwXJjY.

Guest editorial for special section on success and failure in software engineering

Many papers investigate success and failure of software projects from diverse perspectives, leading to a myriad of antecedents, causes, correlates, factors and predictors of success and failure. This body of research has not yet produced a solid, empirically grounded body of evidence enabling actionable practices for increasing success and avoiding failure in software projects. The need for more evidence motivates this special issue, which includes four articles that contribute to our understanding of how software project success and failure relate to topics such as: requirements engineering, user satisfaction, start-up pivots and retrospective discussions. We moreover present a brief systematic review to both situate the accepted articles in existing literature and to explore enduring methodological and conceptual challenges in this area, including developing sound instruments for measuring success, representative sampling without population lists and creating both empirically sound and practically actionable taxonomies of success antecedents.

SECM 2017 Workshop Summary

Summary form only given. The 1st International Workshop on Software Engineering Curricula for Millennials (SECM 2017) aims at bringing students, educators, and prospective employees together, to discuss the demands and challenges of software engineering education for Millennials, in both higher education and professional settings. Millennials are defined as the demographic cohort following Generation X, born between early 1980s and early 2000s. Millennials have been dominating the higher education programs for some time. This cohort has unique needs, learning styles, and skills. They are diverse, collaborative, creative, tech-savvy, and keenly interested in emerging technologies. They drive the growth of the software industry, which itself is in a constant state of flux, with new technologies, techniques, paradigms, and application domains popping up with increasing frequency. Companies quickly adjust to this shifting landscape while trying to cater to the needs of their new hires. What about educators? How should software engineering curricula and educators’ teaching styles adapt to these changes? Perspectives of students, educators, and prospective employees should be heard. Our goal is to bring them together to discuss these challenges in the context of a highly interactive workshop.