Craig Wittenberg

Progress in testing component-based software (abstract only)

Software components enable practical reuse of software parts and amortization of investments over multiple applications. Each part or component is well defined and independently deployable. Composition is the key technique by which systems of software components are constructed. The composition step can be done before or after the delivery of the system. It is this late composition (or at least the possibility of it) which yields the greatest challenges from a testing standpoint. That is, a component-based application may be composed out of parts that were never tested together. Thus the most useful and reliable parts are those which have been tested independently in as many ways as possible. The Component Applications Group in Microsoft Research is developing tools, techniques, and a large component library to enable the development of sophisticated office, home and web-based applications. For the past three and a half years we have been working on two main efforts. First, we have created a prototype of a highly factored (i.e., customizable, flexible, etc.) architecture for the construction of the UI of applications. Our work can be applied to traditional window-ed applications as well as to the look and feel of Web applications. During this effort we have developed a variety of design techniques, two different composition mechanisms, a visual tool for compositions, and have built several application prototypes out of the same set of components. Most of our time has been spent on tools and techniques for building reliable components. Certain pieces of our infrastructure formed the domain in which we tried out our ideas. The first component we tested was one of our composition mechanisms. That was followed by the testing of a dynamic, binary, aspect composition mechanism and of a particularly generic implementation of collection classes. Our well-factored, versioned build system will also be described. All of the results of our work are compatible with COM. The talk will focus on our key lessons in composition, specification, processes, and tools with a particular emphasis on our test harness and our results in testing. A discussion of the last few bugs found in each of several projects should prove intersting. Some comparisons will be made with other projects inside and outside Microsoft. Since we can only claim progress, not perfection, there are still many areas for further research. As an example, we are looking at ways we can use language annotations to simplifying whole classes of problems (e.g., tests for reentrancy). One of the points here is that we can improve our ability to create reliable components by improving the languages used to implement them (like Java has popularized the use of a garbage collector). Another example is that we hope to improve the automation of the sequencing of test cases. Finally, as a tribute to the power of standing on others shoulders, many of the roots of our ideas will be traced to techniques published elsewhere. You might say we only composed together many already good ideas. Our group includes people who developed COM itself (myself and Tony Williams), many people from within Microsoft who have delivered successful component-based products (e.g., in Visual Studio), and world-renowned component-ologist (:-) Clemens Szyperski who wrote Component Software: Beyond Object-Oriented Programming.