Roy Fielding

Two case studies of open source software development: Apache and Mozilla

According to its proponents, open source style software development has the capacity to compete successfully, and perhaps in many cases displace, traditional commercial development methods. In order to begin investigating such claims, we examine data from two major open source projects, the Apache web server and the Mozilla browser. By using email archives of source code change history and problem reports we quantify aspects of developer participation, core team size, code ownership, productivity, defect density, and problem resolution intervals for these OSS projects. We develop several hypotheses by comparing the Apache project with several commercial projects. We then test and refine several of these hypotheses, based on an analysis of Mozilla data. We conclude with thoughts about the prospects for high-performance commercial/open source process hybrids.

A case study of open source software development: the Apache server

According to its proponents, open source style software development has the capacity to compete successfully, and perhaps in many cases displace, traditional commercial development methods. In order to begin investigating such claims, we examine the development process of a major open source application, the Apache web server. By using email archives of source code change history and problem reports we quantify aspects of developer participation, core team size, code ownership, productivity, defect density, and problem resolution interval for this OSS project. This analysis reveals a unique process, which performs well on important measures. We conclude that hybrid forms of development that borrow the most effective techniques from both the OSS and commercial worlds may lead to high performance software processes.

The Apache HTTP Server Project

Most reports of Internet collaboration refer to small scale operations among a few authors or designers. However, several projects have shown that the Internet can also be the locus for large scale collaboration. In these projects, contributors from around the world combine their individual forces and develop a product that rivals those of multibillion dollar corporations. The Apache HTTP Server Project is a case in point. This collaborative software development effort has created a robust, feature-rich HTTP server software package that currently dominates the public Internet market (46 percent compared with 16 percent for Microsoft and 12 percent for Netscape, according to a June 1997 survey published by Netcraft). The software and its source code are free, but Apaches popularity is more often attributed to performance than price. The project is managed by the Apache Group, a geographically distributed group of volunteers who use the Internet and Web to communicate, develop, and distribute the server and its related documentation. In addition, hundreds of users have contributed ideas, code, and documentation to the project.

Shared leadership in the Apache project

documentation. In addition, hundreds of users have contributed ideas, code, and documentation to the project. According to the Netcraft survey of November 1998 [4], the Apache server and its derivatives are used by over 57% of publicly accessible Web sites, more than double its nearest competitor. Unlike most open-source projects, Apache has not been organized around a single person or primary contributor. When the project began in February 1995, the most popular server software on the Web was the public domain HTTP daemon developed by Rob McCool at the National Center for Supercomputing Applications (NCSA). However, development of NCSA httpd had stalled after Rob McCool left, and many Web masters had developed their own extensions and bug fixes that were in need of a common distribution. A small group of these Web masters gathered together via private email for the purpose of coordinating their changes (in the form of “patches”). Brian Behlendorf volunteered the use of his server as a shared information space, providing logins for the eight core developers and hosting a public mailing list for communication. That left two problems to be solved before the development could proceed: decision making and coordination. Apache has always been a multinational project, with core developers located in the U.S., Britain, Canada, Germany, and Italy. Collaboration within the group is hindered by variation in work schedules and the effect of cross-Atlantic network latency. Each Apache Group volunteer has (at least) one other “real” job, usually related to either Web services or protocol research. We collaborate on producing and supporting the Apache server out of enlightened self-interest: by pooling our efforts, the resulting product is much more functional and robust than anything we could have produced alone. However, this also means that none of us can devote large blocks of time to the project. In order for the project to succeed, our development process (the procedures by which we make decisions and coordinate our efforts) had to reflect this globally distributed and volunteer organizational environment. There was no Apache CEO, president, or manager to turn to for making decisions. Instead, we needed to determine group consensus, without using synchronous communication, and in a way that would interfere as little as possible with the project progress. What we devised was a system of voting via email that was based on minimal quorum consensus. Each independent developer could vote on any issue facing the project by sendT he Apache Project [1] is a collaborative software development effort aimed at

Maintaining distributed hypertext infostructures: welcome to MOMspider's Web

Abstract Most documents made available on the World-Wide Web can be considered part of an infostructure — an information resource database with a specifically designed structure. Infostructures often contain a wide variety of information sources, in the form of interlinked documents at distributed sites, which are maintained by a number of different document owners (usually, but not necessarily, the original document authors). Individual documents may also be shared by multiple infostructures. Since it is rarely static, the content of an infostructure is likely to change over time and may vary from the intended structure. Documents may be moved or deleted, referenced information may change, and hypertext links may be broken. As it grows, an infostructure becomes complex and difficult to maintain. Such maintenance currently relies upon the error logs of each server (often never relayed to the document owners), the complaints of users (often not seen by the actual document maintainers), and periodic manual traversals by each owner of all the webs for which they are responsible. Since thorough manual traversal of a web can be time-consuming and boring, maintenance is rarely or inconsistently performed and the infostructure eventually becomes corrupted. What is needed is an automated means for traversing a web of documents and checking for changes which may require the attention of the human maintainers (owners) of that web. The Multi-Owner Maintenance spider (MOMspider 1 ) has been developed to at least partially solve this maintenance problem. MOMspider can periodically traverse a list of webs (by owner, site, or document tree), check each web for any changes which may require its owners attention, and build a special index document that lists out the attributes and connections of the web in a form that can itself be traversed as a hypertext document. This paper describes the design of MOMspider and how it was influenced by the nature of distributed hypertext maintenance and requirements for the good behavior of any web-traversing robot. It also includes discussion of the efficiency requirements for maintaining world-wide webs and proposed changes to HTML and HTTP to support distributed maintenance. The paper concludes with a short description of MOMspiders future and pointers to its freeware distribution site.

Software architecture in an open source world

Summary form only given. In spite of the hype and hysteria surrounding open source software development, there is very little that can be said of open source in general. Open source projects range in scope from the miniscule, such as the thousands of non-maintained code dumps left behind at the end of class projects, dissertations, and failed commercial ventures, to the truly international, with thousands of developers collaborating, directly or indirectly, on a common platform. One characteristic that is shared by the largest and most successful open source projects, however, is a software architecture designed to promote anarchic collaboration through extensions while at the same time preserving centralized control over the interfaces. This paper features a survey of the state-of-the-practice in open source development in regards to software architecture, with particular emphasis on the modular extensibility interfaces within several of the most successful projects, including Apache httpd, Eclipse, Mozilla Firefox, Linux kernel, and the World Wide Web (which few people recognize as an open source project in itself). These projects fall under the general category of collaborative open source software development, which emphasizes community aspects of software engineering in order to compensate for the often-volunteer nature of core developers and take advantage of the scalability obtainable through Internet-based virtual organizations.

Working Group on Design Methods and Processes

A working group met to consider how the SE and HCI communities can learn from each others work on design methods and process. The group identified shared research needs, examined the gaps between the two communities and discussed how those gaps can be bridged, and explored some commonalities between recent research in (Software) Process and (CSCW) Workflow. This report summarizes the issues considered by the group.