Mark Staples

An exploratory study of why organizations do not adopt CMMI

This paper explores why organizations do not adopt CMMI (Capability Maturity Model Integration), by analysing two months of sales data collected by an Australian company selling CMMI appraisal and improvement services. The most frequent reasons given by organizations were: the organization was small; the services were too costly, the organization had no time, and the organization was using another SPI approach. Overall, we found small organizations not adopting CMMI tend to say that adopting it would be infeasible, but do not say it would be unbeneficial. We comment on the significance of our findings and research method for SPI research.

Evaluating guidelines for reporting empirical software engineering studies

BackgroundSeveral researchers have criticized the standards of performing and reporting empirical studies in software engineering. In order to address this problem, Jedlitschka and Pfahl have produced reporting guidelines for controlled experiments in software engineering. They pointed out that their guidelines needed evaluation. We agree that guidelines need to be evaluated before they can be widely adopted.AimThe aim of this paper is to present the method we used to evaluate the guidelines and report the results of our evaluation exercise. We suggest our evaluation process may be of more general use if reporting guidelines for other types of empirical study are developed.MethodWe used a reading method inspired by perspective-based and checklist-based reviews to perform a theoretical evaluation of the guidelines. The perspectives used were: Researcher, Practitioner/Consultant, Meta-analyst, Replicator, Reviewer and Author. Apart from the Author perspective, the reviews were based on a set of questions derived by brainstorming. A separate review was performed for each perspective. The review using the Author perspective considered each section of the guidelines sequentially.ResultsThe reviews detected 44 issues where the guidelines would benefit from amendment or clarification and 8 defects.ConclusionsReporting guidelines need to specify what information goes into what section and avoid excessive duplication. The current guidelines need to be revised and then subjected to further theoretical and empirical validation. Perspective-based checklists are a useful validation method but the practitioner/consultant perspective presents difficulties.Categories and Subject DescriptorsK.6.3 [Software Engineering]: Software Management—Software process.General TermsManagement, Experimentation.

Evaluating guidelines for empirical software engineering studies

Background. Several researchers have criticized the standards of performing and reporting empirical studies in software engineering. In order to address this problem, Andreas Jedlitschka and Dietmar Pfahl have produced reporting guidelines for controlled experiments in software engineering. They pointed out that their guidelines needed evaluation. We agree that guidelines need to be evaluated before they can be widely adopted. If guidelines are flawed, they will cause more problems that they solve.Aim. The aim of this paper is to present the method we used to evaluate the guidelines and report the results of our evaluation exercise. We suggest our evaluation process may be of more general use if reporting guidelines for other types of empirical study are developed.Method. We used perspective-based inspections to perform a theoretical evaluation of the guidelines. A separate inspection was performed for each perspective. The perspectives used were: Researcher, Practitioner/Consultant, Meta-analyst, Replicator, Reviewer and Author. Apart from the Author perspective, the inspections were based on a set of questions derived by brainstorming. The inspection using the Author perspective reviewed each section of the guidelines sequentially. Results. The question-based perspective inspections detected 42 issues where the guidelines would benefit from amendment or clarification and 8 defects.Conclusions. Reporting guidelines need to specify what information goes into what section and avoid excessive duplication. Software engineering researchers need to be cautious about adopting reporting guidelines that differ from those used by other disciplines. The current guidelines need to be revised and the revised guidelines need to be subjected to further theoretical and empirical validation. Perspective-based inspection is a useful validation method but the practitioner/consultant perspective presents difficulties.

Composing enterprise mashup components and services using architecture integration patterns

Enterprise mashups leverage various source of information to compose new situational applications. The architecture of such applications must address integration issues: it needs to deal with heterogeneous local and/or public data sources, and build value-added applications on existing corporate IT systems. In this paper, we leverage enterprise architecture integration patterns to compose reusable mashup components. We present a service oriented architecture that addresses reusability and integration needs for building enterprise mashup applications. Key techniques to customize this architecture are developed for mashups with themed data on location maps. The usage of this architecture is illustrated by a property valuation application derived from a real-world scenario. We demonstrate and discuss how this state-of-the-art architecture design method can be applied to enhance the design and development of emerging enterprise mashups.

On Creating Industry-Wide Reference Architectures

Many industries have been developing e-business standards to improve business-to-business interoperability on a mass scale. Most such standards are composed of business data models with some message exchange patterns. Such data-only standards leave a very large interpretation space for the implementation stage at each individual organization. Thus, true industry-wide interoperability is still hard to achieve. In this industry report, we describe our experiences in creating and evaluating reference architectures for the Australian lending industry. To achieve the right level of prescriptiveness, our reference architectures are deliberately non-structural. Instead, they are based on a set of quality-centric architectural rules. We devised new methods for analyzing interoperability and evaluating such industry-level reference architectures. The first reference architecture has now been adopted and achieved positive effects. We also summarize several other lessons we learned, such as the need to align reference architectures with industry structures.

Analysis of Dependencies between Specific Practices in CMMI Maturity Level 2

CMMI contains a collection of Process Areas (PAs), each of which contains many Specific Practices (SPs). However, the CMMI specification does not provide any explicit recommendation about which individual SPs can or should be implemented before other SPs. In this paper we identify dependencies between CMMI SPs in PAs in maturity level 2, and between the PAs. We analyzed the text of the CMMI specification to identify every Work Product (WP) produced and used by every SP in maturity level 2. Our analysis was validated by independent researchers and comparison with an existing dependency analysis shown in CMMI training materials. Our results have significance as a reference model of SP and PA dependencies for both SPI researchers and practitioners. For researchers we have provided an explicit representation of SP and PA dependencies that were previously only implicit in the CMMI specification. For practitioners, our results may provide guidance on the order of implementation of SPs and PAs. Our dependency analysis has limitations in being derived from the text of the CMMI specification – we have no direct evidence that these dependencies are valid in practice.

Overcoming the First Hurdle: Why Organizations Do Not Adopt CMMI

This paper further examines why some software development organizations decide not to adopt CMMI by replicating an earlier Australian study in another country. The study examines data collected from the efforts of three consulting firms to sell a CMMI Level 2 program subsidized by the Malaysian government. The most frequently cited reasons for not adopting CMMI were: the program was too costly; the companies were unsure of the benefits; the organization was too small; and/or the organization had other priorities. The Malaysian study extends and generally supports the Australian study (differences were found in the frequency ordering of reasons and two new reason categories had to be introduced). It also adds to our understanding of CMMI adoption decisions. Based on the results, we conclude that to achieve broader impact in practice, software process improvement (SPI) researchers need to develop a stronger cost-benefit analysis for SPI, recognising it as a business investment rather than just a product or process quality improvement technique, and provide flexible entry options to enable more companies of difference sizes to take the adoption leap.

An Infrastructure for Indexing and Organizing Best Practices

Industry best practices are widely held but not necessarily empirically verified software engineering beliefs. Best practices can be documented in distributed web-based public repositories as pattern catalogues or practice libraries. There is a need to systematically index and organize these practices to enable their better practical use and scientific evaluation. In this paper, we propose a semi-automatic approach to index and organise best practices. A central repository acts as an information overlay on top of other pre-existing resources to facilitate organization, navigation, annotation and meta-analysis while maintaining synchronization with those resources. An initial population of the central repository is automated using Yahoo! contextual search services. The collected data is organized using semantic web technologies so that the data can be more easily shared and used for innovative analyses. A prototype has demonstrated the capability of the approach.

Effects of architecture and technical development process on micro-process

Current software development methodologies (such as agile and RUP) are largely management-centred, macro-process life-cycle models. While they may include some fine-grained micro-process development practices, they usually provide little concrete guidance on appropriate microprocess level day-to-day development activities. The major factors that affect such micro-process activities are not well understood. We propose that software architecture and technical development processes are two major factors. We describe how these two factors affect micro-process activities. We validate our claim by mining micro-processes from two commercial projects and investigating relationships with software architecture and technical development processes.

Critical rationalism and engineering: : ontology

Engineering is often said to be ‘scientific’, but the nature of knowledge in engineering is different to science. Engineering has a different ontological basis—its theories address different entities and are judged by different criteria. In this paper I use Popper’s three worlds ontological framework to propose a model of engineering theories, and provide an abstract logical view of engineering theories analogous to the deductive-nomological view of scientific theories. These models frame three key elements from definitions of engineering: requirements, designs of artefacts, and theories for reasoning about how artefacts will meet requirements. In a subsequent paper I use this ontological basis to explore methodological issues in the growth of engineering knowledge from the perspective of critical rationalism.