Serge Mankovskii

Software Engineering for Self-Adaptive Systems: A Second Research Roadmap

The goal of this roadmap paper is to summarize the state-of-the-art and identify research challenges when developing, deploying and managing self-adaptive software systems. Instead of dealing with a wide range of topics associated with the field, we focus on four essential topics of self-adaptation: design space for self-adaptive solutions, software engineering processes for self-adaptive systems, from centralized to decentralized control, and practical run-time verification & validation for self-adaptive systems. For each topic, we present an overview, suggest future directions, and focus on selected challenges. This paper complements and extends a previous roadmap on software engineering for self-adaptive systems published in 2009 covering a different set of topics, and reflecting in part on the previous paper. This roadmap is one of the many results of the Dagstuhl Seminar 10431 on Software Engineering for Self-Adaptive Systems, which took place in October 2010.

Towards Practical Runtime Verification and Validation of Self-Adaptive Software Systems

Software validation and verification (VV and (ii) present a proposal for including V&V operations explicitly in feedback loops for ensuring the achievement of software self-adaptation goals. Both of these contributions provide valuable starting points for V&V researchers to help advance this field.

Requirements-Driven root cause analysis using markov logic networks

Root cause analysis for software systems is a challenging diagnostic task, due to the complexity emanating from the interactions between system components and the sheer size of logged data. This diagnostic task is usually assisted by human experts who create mental models of the system-at-hand, in order to generate hypotheses and conduct the analysis. In this paper, we propose a root cause analysis framework based on requirement goal models. We consequently use these models to generate a Markov Logic Network that serves as a diagnostic knowledge repository. The network can be trained and used to provide inferences as to why and how a particular failure observation may be explained by collected logged data. The proposed framework improves over existing approaches by handling uncertainty in observations, using natively generated log data, and by providing ranked diagnoses. The framework is illustrated using a test environment based on commercial off-the-shelf software components.

Symptom-based problem determination using log data abstraction

System failures in industry are expensive, and the increasingly stringent requirements on performance and reliability of enterprise systems have made the detection and diagnosis of system failures crucial and challenging. Log files generated at the system runtime are considered to contain the representations of failure symptoms, and thus become one of the most important sources used for system monitoring and failure diagnosis. A number of studies suggest that data mining and machine learning can help in dealing with the vast amount of log data for a complex enterprise system. Log data abstraction techniques have been proposed, but have not been well studied for failure detection and problem determination. In this research, we investigate the effects of using an unsupervised log data abstraction method to aid the supervised learning processes of problem determination. Additionally, we compare the efficiency of associative classification methods for failure diagnosis against Bayesian Learning technique and C4.5 that have been proved good both in documentation classification and failure diagnosis. Our experimental results show that two associative classification methods outperform Naive Bayes and C4.5 when applied on non-abstracted logs, and unsupervised log abstraction helps to improve the performance of log-based problem determination significantly in terms of the precision, F-measure, and efficiency.

Storm prediction in a cloud

Predicting future behavior reliably and efficiently is key for systems that manage virtual services; such systems must be able to balance loads within a cloud environment to ensure that service level agreements are met at a reasonable expense. In principle accurate predictions can be achieved by mining a variety of data sources, which describe the historic behavior of the services, the requirements of the programs running on them, and the evolving demands placed on the cloud by end users. Of particular importance is accurate prediction of maximal loads likely to be observed in the short term. However, standard approaches to modeling system behavior, by analyzing the totality of the observed data, tend to predict average rather than exceptional system behavior and ignore important patterns of change over time. In this paper, we study the ability of a simple multivariate linear regression for forecasting of peak CPU utilization (storms) in an industrial cloud environment. We also propose several modifications to the standard linear regression to adjust it for storm prediction.

DRACA: decision support for root cause analysis and change impact analysis for CMDBs

As business services become increasingly dependent on information technology (IT), it also becomes increasingly important to maximize the decision support for managing IT. Configuration Management Data Bases (CMDBs) store fundamental information about IT systems, such as the systems hardware, software and services. This information can help provide decision support for root cause analysis and change impact analysis. We have worked with our industrial research partner, CA, and with CA customers to identify challenges to the use of CMDBs to semi-automatically solve these problems. In this paper we propose a framework called DRACA (Decision Support for Root Cause Analysis and Change Impact Analysis). This framework mines key facts from the CMDB and in a sequence of three steps combines these facts with incident reports, change reports and expert knowledge, along with temporal information, to construct a probabilistic causality graph. Root causes are predicted and ranked by probabilistically tracing causality edges backwards from incidents to likely causes. Conversely, change impacts can be predicted and ranked by tracing from a proposed change forward along causality edges to locate likely undesirable impacts.

Chapter 18 – Mining Software Logs for Goal-Driven Root Cause Analysis

Root cause analysis for software systems is a challenging diagnostic task owing to the complex interactions between system components, the sheer volume of logged data, and the often partial and incomplete information available for root cause analysis purposes. This diagnostic task is usually performed by human experts who create mental models of the system at hand, generate root cause hypotheses, conduct log analysis, and identify the root causes of an observed system failure. In this chapter, we discuss a root cause analysis framework that is based on goal and antigoal models to represent the relationship between a system behavior or requirement, and the necessary conditions, configurations, properties, constraints, or external actions that affect this particular system behavior. We consequently use these models to generate a Markov logic network that allows probabilistic reasoning, so that conclusions can be reached even with incomplete or partial data and observations. The proposed framework improves on existing approaches by handling uncertainty in observations, using natively generated log data, and by being able to provide ranked diagnoses. The framework is evaluated using a sample application based on commercial off-the-shelf software components.

Analyzing Assembler to Eliminate Dead Functions: An Industrial Experience

Industrial software systems often contain fragments of code that are vestigial, that is, they were created long ago for a specific purpose but are no longer useful within the current design of the system. In this work, we describe how we have adapted some research tools to remove such code, we use a hybrid static analysis approach of both source code and assembler to construct a model of the system, and then use graph querying to detect possible dead functions. Suspected dead functions are then commented out of the source. The system is then rebuilt and run against existing test suites to verify that the removals do not affect the semantics of the system. Finally, we discuss the results of performing this technique on a large and long-lived industrial software system as well as a large open source system.

Event clustering for log reduction and run time system understanding

Large software systems are constantly monitored so that audits can be initiated, once a failure occurs or when maintenance operations are performed. However, the log files are usually sizeable, and require filtering and reduction in order to be processed efficiently. In this paper, we define the concept of the Event Dependency Graph, and we discuss an event filtering and a use case identification technique, that is based on event clustering. This technique can be used to reduce the size of system logs and assist on system analysis and, program understanding.

Managing information overload on large enterprise systems

Managers and operators of modern enterprise IT environments are faced with the daunting task of monitoring hundreds of thousands of interconnected elements, from individual hardware components and virtualized applications, to logical business services. Research results [1] and anecdotal evidence from network administrators indicate that common visualization techniques used to represent networks do not scale well for large dynamic environments and viewers can suffer from information overload. We interviewed managers and operators of an enterprise IT environment regarding the challenges they face when dealing with large and complex systems, how they manage to work around those problems and how well visualization tools support their workflow. Our results show the strategies used by participants to limit the effects of information overload and apply them in the design a new visualization metaphor for navigation on complex environments.