Alan MacCormack

Managing the Sources of Uncertainty: Matching Process and Context in Software Development

There is increasing interest in the literature about the notion of a contingent approach to product development process design. This interest stems from the realization that different types of projects carried out in different environments are likely to require quite different development processes if they are to be successful. Stated more formally, a contingent view implies that the performance impact of different development practices is likely to be mediated by the context in which those practices operate. This article provides evidence to support such a view. Our work examines whether projects in which the development process matches the context achieve superior performance. We focus on two sources of uncertainty that generate challenges for project teams: platform uncertainty, reflecting the uncertainty generated by the amount of new design work that must be undertaken in a project; and market uncertainty, reflecting the uncertainty faced in determining customer requirements for the product under development. We develop hypotheses for how these sources of uncertainty are likely to influence the relationships between a number of specific development practices and performance. We then test these hypotheses using data from a sample of 29 Internet software development projects. Our results provide evidence to support a contingent view of development process design. We show that in projects facing greater uncertainty, investments in architectural design, early technical feedback, and early market feedback have a stronger association with performance. The latter relationships are influenced by the specific sources from which this uncertainty stems: platform uncertainty mediating the impact of early technical feedback and market uncertainty mediating the impact of early market feedback. Our results also indicate that while greater uncertainty is associated with making later changes to a products design, this practice is not associated with performance. Our findings suggest that managers carefully must evaluate both the levels and sources of uncertainty facing a project before designing the most appropriate process for its execution. In particular, they should explore the use of specific development practices based upon their usefulness in resolving the specific types of uncertainty faced. Importantly, these decisions must be made at the start of a project, with purposeful investments to create a process that best matches the context. Reacting to uncertainty ex-post, without such investments in place, is unlikely to prove a successful strategy.

Prize-based contests can provide solutions to computational biology problems

Advances in biotechnology have fuelled the generation of unprecedented quantities of data across the life sciences. However, finding individuals who can address such “big data” problems effectively has become a significant research bottleneck. Historically, prize-based contests have had striking success in attracting unconventional individuals who can solve difficult challenges. To determine whether this approach could solve a real “big data” biologic algorithm problem, we used a complex immunogenomics problem as the basis for a two-week online contest broadcast to participants outside academia and biomedical disciplines. Participants in our contest generated over 600 submissions containing 89 novel computational approaches to the problem. Thirty submissions exceeded the benchmark performance of NIH’s MegaBLAST. The best achieved both greater accuracy and speed (x1000). Here we show the potential of using online prize-based contests to access individuals without domain-specific backgrounds to address big data challenges in life sciences.

Analyzing the Evolution of Large-Scale Software Systems Using Design Structure Matrices and Design Rule Theory: Two Exploratory Cases

Designers have long recognized the value of modularity, but important software modularity principles have remained informal. According to Baldwin and Clarks (2000) design rule theory (DRT) , modular architectures add value to system designs by creating options to improve the system by substituting or experimenting on individual modules. In this paper, we examine the design evolution of two software product platforms through the modeling lens of DRT and design structure matrices (DSMs). We show that DSM models and DRT precisely explain how real- world modularization activities in one case allowed for different rates of evolution in different software modules and in another case conferred distinct strategic advantages on a firm by permitting substitution of an at-risk software module without substantial change to the rest of the system. Our results provide positive evidence that DSM and DRT can inform important aspects of large-scale software structure and evolution, having the potential to guide software architecture design activities.

Visualizing and Measuring Enterprise Application Architecture: An Exploratory Telecom Case

We test a method for visualizing and measuring enterprise application architectures. The method was designed and previously used to reveal the hidden internal architectural structure of software applications. The focus of this paper is to test if it can also uncover new facts about the applications and their relationships in an enterprise architecture, i.e., if the method can reveal the hidden external structure between software applications. Our test uses data from a large international telecom company. In total, we analyzed 103 applications and 243 dependencies. Results show that the enterprise application structure can be classified as a core-periphery architecture with a propagation cost of 25%, core size of 34%, and architecture flow through of 64%. These findings suggest that the method could be effective in uncovering the hidden structure of an enterprise application architecture.

Hidden Structure: Using Network Methods to Map System Architecture

In this paper, we describe an operational methodology for characterizing the architecture of complex technical systems and demonstrate its application to a large sample of software releases. Our methodology is based upon directed network graphs, which allows us to identify all of the direct and indirect linkages between the components in a system. We use this approach to define three fundamental architectural patterns, which we label core–periphery, multi-core, and hierarchical. Applying our methodology to a sample of 1286 software releases from 17 applications, we find that the majority of releases possess a “core–periphery” structure. This architecture is characterized by a single dominant cyclic group of components (the “Core”) that is large relative to the system as a whole as well as to other cyclic groups in the system. We show that the size of the Core varies widely, even for systems that perform the same function. These differences appear to be associated with different models of development – open, distributed organizations develop systems with smaller Cores, while closed, co-located organizations develop systems with larger Cores. Our findings establish some “stylized facts” about the fine-grained structure of large, real-world technical systems, serving as a point of departure for future empirical work.

Do You Need a New Product-Development Strategy?

OVERVIEW: Many firms rely on a single new-product development process for all projects. But designing new products for different business contexts requires that a firm deploy different new-product development processes. Products designed for stable and mature end-user markets require a process optimized for control and efficiency. In contrast, first-of-a-kind “breakthrough” products require a more emergent process that aims to discover whether there is any market to be served in the first place. Applying a uniform “best-practice” process to all development efforts ignores the major differences between these projects and may result in missed opportunities. This article describes a framework to address this problem, allowing a firm to better align the design of its development processes to the specific aims of individual projects. We illustrate this framework with examples from Hewlett-Packard, a large, diversified electronics firm that has successfully piloted this new approach across multiple business units.

Visualizing and Measuring Enterprise Architecture: An Exploratory BioPharma Case

We test a method that was designed and used previously to reveal the hidden internal architectural structure of software systems. The focus of this paper is to test if it can also uncover new facts about the components and their relationships in an enterprise architecture, i.e., if the method can reveal the hidden external structure between architectural components. Our test uses data from a biopharmaceutical company. In total, we analyzed 407 components and 1,157 dependencies. Results show that the enterprise structure can be classified as a core-periphery architecture with a propagation cost of 23%, core size of 32%, and architecture flow through of 67%. We also found that business components can be classified as control elements, infrastructure components as shared, and software applications as belonging to the core. These findings suggest that the method could be effective in uncovering the hidden structure of an enterprise architecture.

Visualizing and Measuring Software Portfolio Architecture: A Flexibility Analysis

In this paper, we test a Design Structure Matrix (DSM) based method for visualizing and measuring software portfolio architectures, and use our measures to predict the costs of architectural change ...

Exploring the Relationship Between Architecture Coupling and Software Vulnerabilities

Employing software metrics, such as size and complexity, for predicting defects has been given a lot of attention over the years and proven very useful. However, the few studies looking at software architecture and vulnerabilities are limited in scope and findings. We explore the relationship between software vulnerabilities and component metrics (like code churn and cyclomatic complexity), as well as architecture coupling metrics (direct, indirect, and cyclic coupling). Our case is based on the Google Chromium project, an open source project that has not been studied for this topic yet. Our findings show a strong relationship between vulnerabilities and both component level metrics and architecture coupling metrics. 68% of the files associated with a vulnerability are cyclically coupled, compared to 43% of the non-vulnerable files. Our best regression model is a combination of low commenting, high code churn, high direct fan-out within the main cyclic group, and high direct fan-in outside of the main cyclic group.

Aligning Partnering Strategy, Partner Governance and Partner Management in R&D Projects: The Impact of Contract Choice

Firms increasingly look outside their organizational boundaries to identify partners that can improve the effectiveness of RD whereas in others they are used to improve the performance or quality of the final product. How should these variations in partnering strategy impact the governance and management choices made within projects? We examine this question using data on 172 RD and second, how this choice subsequently affects the relationship between partner integration and partnering performance in the project. The performance measures studied here focus specifically on partnering contributions to project costs and product quality. Our results indicate that the choice of contract is a function of the partnering strategy for a project, more flexible contracts being preferred in projects that seek long-run capability-based benefits and have broader scope of partnering relationships. These choices, in turn, impact the benefits associated with partner integration; while higher levels of partner integration are always associated with higher costs, they enable partners to contribute toward higher product quality only in projects that use more flexible contracts. We further show that in projects where partnering strategy and contract choice are misaligned, partner integration has no impact on product quality despite increasing project costs. We conclude by discussing the implications of our findings and suggest new directions for future research.