Liming Zhu

A framework for classifying and comparing software architecture evaluation methods

Software architecture evaluation has been proposed as a means to achieve quality attributes such as maintainability and reliability in a system. The objective of the evaluation is to assess whether or not the architecture lead to the desired quality attributes. Recently, there have been a number of evaluation methods proposed. There is, however, little consensus on the technical and nontechnical issues that a method should comprehensively address and which of the existing methods is most suitable for a particular issue. We present a set of commonly known but informally described features of an evaluation method and organizes them within a framework that should offer guidance on the choice of the most appropriate method for an evaluation exercise. We use this framework to characterise eight SA evaluation methods.

Tradeoff and Sensitivity Analysis in Software Architecture Evaluation Using Analytic Hierarchy Process

Software architecture evaluation involves evaluating different architecture design alternatives against multiple quality-attributes. These attributes typically have intrinsic conflicts and must be considered simultaneously in order to reach a final design decision. AHP (Analytic Hierarchy Process), an important decision making technique, has been leveraged to resolve such conflicts. AHP can help provide an overall ranking of design alternatives. However it lacks the capability to explicitly identify the exact tradeoffs being made and the relative size of these tradeoffs. Moreover, the ranking produced can be sensitive such that the smallest change in intermediate priority weights can alter the final order of design alternatives. In this paper, we propose several in-depth analysis techniques applicable to AHP to identify critical tradeoffs and sensitive points in the decision process. We apply our method to an example of a real-world distributed architecture presented in the literature. The results are promising in that they make important decision consequences explicit in terms of key design tradeoffs and the architectures capability to handle future quality attribute changes. These expose critical decisions which are otherwise too subtle to be detected in standard AHP results.

Evaluating guidelines for reporting empirical software engineering studies

BackgroundSeveral researchers have criticized the standards of performing and reporting empirical studies in software engineering. In order to address this problem, Jedlitschka and Pfahl have produced reporting guidelines for controlled experiments in software engineering. They pointed out that their guidelines needed evaluation. We agree that guidelines need to be evaluated before they can be widely adopted.AimThe aim of this paper is to present the method we used to evaluate the guidelines and report the results of our evaluation exercise. We suggest our evaluation process may be of more general use if reporting guidelines for other types of empirical study are developed.MethodWe used a reading method inspired by perspective-based and checklist-based reviews to perform a theoretical evaluation of the guidelines. The perspectives used were: Researcher, Practitioner/Consultant, Meta-analyst, Replicator, Reviewer and Author. Apart from the Author perspective, the reviews were based on a set of questions derived by brainstorming. A separate review was performed for each perspective. The review using the Author perspective considered each section of the guidelines sequentially.ResultsThe reviews detected 44 issues where the guidelines would benefit from amendment or clarification and 8 defects.ConclusionsReporting guidelines need to specify what information goes into what section and avoid excessive duplication. The current guidelines need to be revised and then subjected to further theoretical and empirical validation. Perspective-based checklists are a useful validation method but the practitioner/consultant perspective presents difficulties.Categories and Subject DescriptorsK.6.3 [Software Engineering]: Software Management—Software process.General TermsManagement, Experimentation.

UML Profiles for Design Decisions and Non-Functional Requirements

A software architecture is composed of a collection of design decisions. Each design decision helps or hinders certain Non-Functional Requirements (NFR). Current software architecture views focus on expressing components and connectors in the system. Design decisions and their relationships with non-functional requirements are often captured in separate design documentation, not explicitly expressed in any views. This disassociation makes architecture comprehension and architecture evolution harder. In this paper, we propose a UML profile for modeling design decisions and an associated UML profile for modeling non-functional requirements in a generic way. The two UML profiles treat design decisions and nonfunctional requirements as first-class elements. Modeled design decisions always refer to existing architectural elements and thus maintain traceability between the two. We provide a mechanism for checking consistency over this traceability. An exemplar is given as a way to demonstrate the feasibility of our approach.

Mining patterns to support software architecture evaluation

In this paper, we present an approach to improve the software architecture evaluation process by systematically extracting and appropriately documenting architecturally significant information from software architecture and design patterns; we are interested in only two pieces of information found in software patterns: general scenarios and architectural tactics. General scenarios distilled from patterns not only assist stakeholders in developing concrete scenarios during a scenario-based architecture evaluation, but can also help an architect select and calibrate a quality attribute reasoning framework. Architectural tactics in patterns are used as a means of manipulating independent parameters in the reasoning framework to achieve the desired quality. Moreover, we believe if we use general scenarios and tactics extracted from patterns in an architectural evaluation, the results of that evaluation can be used as an evidence to validate the patterns claim with respect to the quality attributes. We demonstrate our approach by using EJB architecture usage patterns. We contend that this approach can be used to analyze and validate any architecture pattern.

Resource-Oriented Architecture for Business Processes

REpresentational State Transfer (REST) is the set of design principles behind the World Wide Web (WWW). REST treats all entities in the world as link-connected resources, and supports a resource-oriented architecture (ROA) for the design of applications. REST and ROA are responsible for many of the desirable quality attributes achieved in the WWW, such as loose-coupling (better adaptability) and interoperability. However, many exiting Web-based or service-oriented applications (WSDL/SOAP-based) only use WWW/HTTP as a tunneling protocol or abuse URL and POX (Plain Old XML) by encoding method semantics in them. These applications use fine-grained remote procedure calls (RPC), breaking REST/ROA principles. We observe two kinds of challenges: 1) conceptually modelling process-intensive applications using a ROA promoted by the REST principles; and 2) practically decomposing a workflow-based business process into distributed, dynamic and RESTful process fragments. In this paper, we propose a ROA for business processes following the RESTful principles. We evaluate our approach by comparing it with current SOAP/WSDL/BPEL-driven approaches in terms of feasibility, process visibility, interoperability, and adaptability.

Performance Prediction of Service-Oriented Applications based on an Enterprise Service Bus

An enterprise service bus (ESB) is a standards-based integration platform that combines messaging, web services, data transformation, and intelligent routing in a highly distributed environment. The ESB has been adopted as a key component of SOA infrastructures. For SOA implementations with large number of users, services, or traffic, maintaining the necessary performance levels of applications integrated using an ESB presents a substantial challenge, both to the architects who design the infrastructure as well as to IT professionals who are responsible for administration. In this paper, we develop a performance model for analyzing and predicting the runtime performance of service applications composed on a COTS ESB platform. Our approach utilizes benchmarking techniques to measure primitive performance overheads of service routing activities in the ESB. The performance characteristics of the ESB and services running on the ESB are modeled in a queuing network, which facilitates the performance prediction of service oriented applications. This model is validated by an example ESB based service application modeled from real world loan broking business application.

The Blockchain as a Software Connector

Blockchain is an emerging technology for decentralized and transactional data sharing across a large network of untrusted participants. It enables new forms of distributed software architectures, where components can find agreements on their shared states without trusting a central integration point or any particular participating components. Considering the blockchain as a software connector helps make explicitly important architectural considerations on the resulting performance and quality attributes (for example, security, privacy, scalability and sustainability) of the system. Based on our experience in several projects using blockchain, in this paper we provide rationales to support the architectural decision on whether to employ a decentralized blockchain as opposed to other software solutions, like traditional shared data storage. Additionally, we explore specific implications of using the blockchain as a software connector including design trade-offs regarding quality attributes.

An exploratory study of groupware support for distributed software architecture evaluation process

Software architecture evaluation is an effective means of addressing quality related issues quite early in the software development lifecycle. Scenario-based approaches to evaluate architecture usually involve a large number of stakeholders, who need to be collocated for evaluation sessions. Collocating a large number of stakeholders is an expensive and time-consuming exercise, which may prove to be a hurdle in the wide-spread adoption of architectural evaluation practices. Drawing upon the successful introduction of groupware applications to support geographically distributed teams in software inspection, and requirements engineering disciplines, we propose the concept of distributed architectural evaluation using Internet-based collaborative technologies. This paper illustrates the methodology of a pilot study to assess the viability of a larger experiment intended to investigate the feasibility of groupware support for distributed software architecture evaluation. In addition, the results of the pilot study provide some interesting findings on the viability of groupware-supported software architectural evaluation process.

Tool support for just-in-time architecture reconstruction and evaluation: an experience report

The need for software architecture evaluation has drawn considerable attention in recent years. In practice, this is a challenging exercise for two main reasons. First, in deployed projects, software architecture documentation is often not readily available, and may not be a correct representation of the as built architecture. Second, large software systems have numerous potential views of the various architecturally significant structures in the system. In this paper we assess the capabilities of software reverse engineering and architecture reconstruction tools to support just-in-time architecture reconstruction. If an applications architecture can be reconstructed efficiently, this could promote more effective architecture reviews and evaluations. We describe our experiences in leveraging multiple reconstruction tools and how these guided the choice of design artifacts to construct. We discovered that the tools complemented each other in identifying reconstruction scope, critical architectural elements, potential design irregularities and creating useful architectural views for different evaluation tasks. With the help of these tools, the reconstruction and evaluation effort was significantly streamlined and productive. Finally, we also report some potential improvements these tools could make.