Lars Grunske

Software Architecture Optimization Methods: A Systematic Literature Review

Due to significant industrial demands toward software systems with increasing complexity and challenging quality requirements, software architecture design has become an important development activity and the research domain is rapidly evolving. In the last decades, software architecture optimization methods, which aim to automate the search for an optimal architecture design with respect to a (set of) quality attribute(s), have proliferated. However, the reported results are fragmented over different research communities, multiple system domains, and multiple quality attributes. To integrate the existing research results, we have performed a systematic literature review and analyzed the results of 188 research papers from the different research communities. Based on this survey, a taxonomy has been created which is used to classify the existing research. Furthermore, the systematic analysis of the research literature provided in this review aims to help the research community in consolidating the existing research efforts and deriving a research agenda for future developments.

An Approach to Forecasting QoS Attributes of Web Services Based on ARIMA and GARCH Models

Availability of several web services having a similar functionality has led to using quality of service (QoS) attributes to support services selection and management. To improve these operations and be performed proactively, time series ARIMA models have been used to forecast the future QoS values. However, the problem is that in this extremely dynamic context the observed QoS measures are characterized by a high volatility and time-varying variation to the extent that existing ARIMA models cannot guarantee accurate QoS forecasting where these models are based on a homogeneity (constant variation over time) assumption, which can introduce critical problems such as proactively selecting a wrong service and triggering unrequired adaptations and thus leading to follow-up failures and increased costs. To address this limitation, we propose a forecasting approach that integrates ARIMA and GARCH models to be able to capture the QoS attributes volatility and provide accurate forecasts. Using QoS datasets of real-world web services we evaluate the accuracy and performance aspects of the proposed approach. Results show that the proposed approach outperforms the popular existing ARIMA models and improves the forecasting accuracy of QoS measures and violations by on average 28.7% and 15.3% respectively.

An approach to software reliability prediction based on time series modeling

Reliability is the key factor for software system quality. Several models have been introduced to estimate and predict reliability based on results of software testing activities. Software Reliability Growth Models (SRGMs) are considered the most commonly used to achieve this goal. Over the past decades, many researchers have discussed SRGMs assumptions, applicability, and predictability. They have concluded that SRGMs have many shortcomings related to their unrealistic assumptions, environment-dependent applicability, and questionable predictability. Several approaches based on non-parametric statistics, Bayesian networks, and machine learning methods have been proposed in the literature. Based on their theoretical nature, however, they cannot completely address the SRGMs limitations. Consequently, addressing these shortcomings is still a very crucial task in order to provide reliable software systems. This paper presents a well-established prediction approach based on time series ARIMA (Autoregressive Integrated Moving Average) modeling as an alternative solution to address the SRGMs limitations and provide more accurate reliability prediction. Using real-life data sets on software failures, the accuracy of the proposed approach is evaluated and compared to popular existing approaches.

Using Models at Runtime to Address Assurance for Self-Adaptive Systems

A self-adaptive software system modifies its behavior at runtime in response to changes within the system or in its execution environment. The ful- fillment of the system requirements needs to be guaranteed even in the presence of adverse conditions and adaptations. Thus, a key challenge for self-adaptive software systems is assurance. Traditionally, confidence in the correctness of a system is gained through a variety of activities and processes performed at de- velopment time, such as design analysis and testing. In the presence of self- adaptation, however, some of the assurance tasks may need to be performed at runtime. This need calls for the development of techniques that enable contin- uous assurance throughout the software life cycle. Fundamental to the develop- ment of runtime assurance techniques is research into the use of models at runtime

An automated approach to forecasting QoS attributes based on linear and non-linear time series modeling

Predicting future values of Quality of Service (QoS) attributes can assist in the control of software intensive systems by preventing QoS violations before they happen. Currently, many approaches prefer Autoregressive Integrated Moving Average (ARIMA) models for this task, and assume the QoS attributes behavior can be linearly modeled. However, the analysis of real QoS datasets shows that they are characterized by a highly dynamic and mostly nonlinear behavior to the extent that existing ARIMA models cannot guarantee accurate QoS forecasting, which can introduce crucial problems such as proactively triggering unrequired adaptations and thus leading to follow-up failures and increased costs. To address this limitation, we propose an automated forecasting approach that integrates linear and nonlinear time series models and automatically, without human intervention, selects and constructs the best suitable forecasting model to fit the QoS attributes dynamic behavior. Using real-world QoS datasets of 800 web services we evaluate the applicability, accuracy, and performance aspects of the proposed approach, and results show that the approach outperforms the popular existing ARIMA models and improves the forecasting accuracy by on average 35.4%.

Architecture-based reliability evaluation under uncertainty

The accuracy of architecture-based reliability evaluations depends on a number of parameters that need to be estimated, such as environmental factors or system usage. Researchers have tackled this problem by including uncertainties in architecture evaluation models and solving them analytically and with simulations. The usual assumption is that the input parameter distributions are normal, and that it is sufficient to report the attributes that describe the system in terms of the mean and variance of the attribute. In this work, we introduce a simulation-based approach that can accommodate a diverse set of parameter range distributions, self-regulate the number of architecture evaluations to the desired significance level and reports the desired percentiles of the values which ultimately characterise a specific quality attribute of the system. We include a case study which illustrates the flexibility of this approach using the evaluation of system reliability.

Aligning Qualitative, Real-Time, and Probabilistic Property Specification Patterns Using a Structured English Grammar

Formal methods offer an effective means to assert the correctness of software systems through mathematical reasoning. However, the need to formulate system properties in a purely mathematical fashion can create pragmatic barriers to the application of these techniques. For this reason, Dwyer et al. invented property specification patterns which is a system of recurring solutions to deal with the temporal intricacies that would make the construction of reactive systems very hard otherwise. Today, property specification patterns provide general rules that help practitioners to qualify order and occurrence, to quantify time bounds, and to express probabilities of events. Nevertheless, a comprehensive framework combining qualitative, real-time, and probabilistic property specification patterns has remained elusive. The benefits of such a framework are twofold. First, it would remove the distinction between qualitative and quantitative aspects of events; and second, it would provide a structure to systematically discover new property specification patterns. In this paper, we report on such a framework and present a unified catalogue that combines all known plus 40 newly identified or extended patterns. We also offer a natural language front-end to map patterns to a temporal logic of choice. To demonstrate the virtue of this new framework, we applied it to a variety of industrial requirements, and use PSPWizard, a tool specifically developed to work with our unified pattern catalogue, to automatically render concrete instances of property specification patterns to formulae of an underlying temporal logic of choice.

A learning-to-rank based fault localization approach using likely invariants

Debugging is a costly process that consumes much of developer time and energy. To help reduce debugging effort, many studies have proposed various fault localization approaches. These approaches take as input a set of test cases (some failing, some passing) and produce a ranked list of program elements that are likely to be the root cause of the failures (i.e., failing test cases). In this work, we propose Savant, a new fault localization approach that employs a learning-to-rank strategy, using likely invariant diffs and suspiciousness scores as features, to rank methods based on their likelihood to be a root cause of a failure. Savant has four steps: method clustering & test case selection, invariant mining, feature extraction, and method ranking. At the end of these four steps, Savant produces a short ranked list of potentially buggy methods. We have evaluated Savant on 357 real-life bugs from 5 programs from the Defects4J benchmark. Out of these bugs, averaging over 100 repeated trials with different seeds to randomly break ties, we find that on average Savant can identify correct buggy methods for 63.03, 101.72, and 122 bugs at top 1, 3, and 5 positions in the ranked lists that Savant produces. We have compared Savant against several state-of-the-art fault localization baselines that work on program spectra. We show that Savant can successfully locate 57.73%, 56.69%, and 43.13% more bugs at top 1, top 3, and top 5 positions than the best performing baseline, respectively.

Dimensions and Metrics for Evaluating Recommendation Systems

Recommendation systems support users and developers of various computer and software systems to overcome information overload, perform information discovery tasks, and approximate computation, among others. They have recently become popular and have attracted a wide variety of application scenarios ranging from business process modeling to source code manipulation. Due to this wide variety of application domains, different approaches and metrics have been adopted for their evaluation. In this chapter, we review a range of evaluation metrics and measures as well as some approaches used for evaluating recommendation systems. The metrics presented in this chapter are grouped under sixteen different dimensions, e.g., correctness, novelty, coverage. We review these metrics according to the dimensions to which they correspond. A brief overview of approaches to comprehensive evaluation using collections of recommendation system dimensions and associated metrics is presented. We also provide suggestions for key future research and practice directions.

Model-based performance analysis of software architectures under uncertainty

Performance analysis is often conducted before achieving full knowledge of a software system, in other words under a certain degree of uncertainty. Uncertainty is particularly critical in the performance domain when it relates to values of parameters such as workload, operational profile, resource demand of services, service time of hardware devices, etc. The goal of this paper is to explicitly consider uncertainty in the performance modelling and analysis process. In particular, we use probabilistic formulation of parameter uncertainties and present a Monte Carlo simulation-based approach to systematically assess the robustness of an architectural model despite its uncertainty. In case of unsatisfactory results, we introduce refactoring actions aimed at generating new software architectural models that better tolerate the uncertainty of parameters. The proposed approach is illustrated on a case study from the e-Health domain.