Michael Vierhauser

Flexible and scalable consistency checking on product line variability models

The complexity of product line variability models makes it hard to maintain their consistency over time regardless of the modeling approach used. Engineers thus need support for detecting and resolving inconsistencies. We describe experiences of applying a tool-supported approach for incremental consistency checking on variability models. Our approach significantly improves the overall performance and scalability compared to batch-oriented techniques and allows providing immediate feedback to modelers. It is extensible as new consistency constraints can easily be added. Furthermore, the approach is flexible as it is not limited to variability models and it also checks the consistency of the models with the underlying code base of the product line. We report the results of a thorough evaluation based on real-world product line models and discuss lessons learned.

ReMinds : A flexible runtime monitoring framework for systems of systems

Abstract Many software-intensive systems today can be characterized as systems of systems (SoS) comprising complex, interrelated, and heterogeneous systems. The behavior of SoS often only emerges at runtime due to complex interactions between the involved systems and their environment. It is thus necessary to determine unexpected behavior by monitoring SoS at runtime, i.e., collecting and analyzing events and data at different layers and levels of granularity. Existing monitoring approaches are often limited to individual systems, particular architectural styles, or technologies. In this paper we thus derive challenges for monitoring SoS based on an industrial case. We present a flexible framework adaptable to different system architectures and technologies. We discuss its capabilities for instrumenting systems, collecting and persisting events and data, checking constraints on events and data, and visualizing the systems’ behavior to users. We demonstrate the framework’s flexibility by tailoring and applying it to an industrial SoS and assessing its performance and scalability. Our results show that the framework is flexible and scalable for monitoring an industrial SoS with realistic event loads.

A case study on testing, commissioning, and operation of very-large-scale software systems

An increasing number of software systems today are very-large-scale software systems (VLSS) with system-of-systems (SoS) architectures. Due to their heterogeneity and complexity VLSS are difficult to understand and analyze, which results in various challenges for development and evolution. Existing software engineering processes, methods, and tools do not sufficiently address the characteristics of VLSS. Also, there are only a few empirical studies on software engineering for VLSS. We report on results of an exploratory case study involving engineers and technical project managers of an industrial automation VLSS for metallurgical plants. The paper provides empirical evidence on how VLSS are tested, commissioned, and operated in practice. The paper discusses practical challenges and reports industrial requirements regarding process and tool support. In particular, software processes and tools need to provide general guidance at the VLSS level as well as specific methods and tools for systems that are part of the VLSS. Processes and tools need to support multi-disciplinary engineering across system boundaries. Furthermore, managing variability and evolution is success-critical in VLSS verification and validation.

A Flexible Framework for Runtime Monitoring of System-of-Systems Architectures

Many software systems today have system-of systems (SoS) architectures comprising interrelated and heterogeneous systems, which are developed by multiple teams and companies. Such systems emerge gradually and it is hard to analyze or predict their behavior due to their scale and complexity. In particular, certain behavior only emerges at runtime due to complex interactions between the involved systems and their environment. Monitoring the behavior of SoS at runtime is thus essential during development and evolution. However, existing monitoring approaches are often limited to particular architectural styles or technologies and are thus hard to apply in SoS architectures. In this paper we first analyze the challenges for monitoring SoS based on an industrial SoS for the automation of metallurgical plants. We then propose a flexible framework for monitoring heterogeneous systems within a SoS. We demonstrate its feasibility by applying it to two systems of an industrial SoS. We also report results of an evaluation assessing the frameworks performance and scalability.

Evolution in dynamic software product lines: challenges and perspectives

In many domains systems need to run continuously and cannot be shut down for reconfiguration or maintenance tasks. Cyber-physical or cloud-based systems, for instance, thus often provide means to support their adaptation at runtime. The required flexibility and adaptability of systems suggests the application of Software Product Line (spl) principles to manage their variability and to support their reconfiguration. Specifically, Dynamic Software Product Lines (dspl) have been proposed to support the management and binding of variability at runtime. While spl evolution has been widely studied, it has so far not been investigated in detail in a dspl context. Variability models that are used in a dspl have to co-evolve and be kept consistent with the systems they represent to support reconfiguration even after changes to the systems at runtime. In this short paper we present a classification of the required operations for jointly evolving problem and solution space in a dspl. We analyze the impact of such operations on the consistency of a dspl and propose an approach to deal with the described issues. We describe a runtime monitoring system used in the domain of industrial automation software as an example of a dspl evolving at runtime to motivate and explain our work.

Developing a DSL-Based Approach for Event-Based Monitoring of Systems of Systems: Experiences and Lessons Learned (E)

Complex software-intensive systems are often described as systems of systems (SoS) comprising heterogeneous architectural elements. As SoS behavior fully emerges during operation only, runtime monitoring is needed to detect deviations from requirements. Today, diverse approaches exist to define and check runtime behavior and performance characteristics. However, existing approaches often focus on specific types of systems and address certain kinds of checks, thus impeding their use in industrial SoS. Furthermore, as many SoS need to run continuously for long periods, the dynamic definition and deployment of constraints needs to be supported. In this paper we describe experiences of developing and applying a DSL-based approach for monitoring an SoS in the domain of industrial automation software. We evaluate both the expressiveness of our DSL as well as the scalability of the constraint checker. We also describe lessons learned.

A comparison framework for runtime monitoring approaches

We present a comparison framework for runtime monitoring approaches.We demonstrate the usefulness of our framework.We discuss perspectives for researchers working in the area of monitoring. The full behavior of complex software systems often only emerges during operation. They thus need to be monitored at run time to check that they adhere to their requirements. Diverse runtime monitoring approaches have been developed in various domains and for different purposes. Their sheer number and heterogeneity, however, make it hard to find the right approach for a specific application or purpose. The aim of our research therefore was to develop a comparison framework for runtime monitoring approaches. Our framework is based on an analysis of the literature and existing taxonomies for monitoring languages and patterns. We use examples from existing monitoring approaches to explain the framework. We demonstrate its usefulness by applying it to 32 existing approaches and by comparing 3 selected approaches in the light of different monitoring scenarios. We also discuss perspectives for researchers.

Requirements monitoring frameworks

Analyze the characteristics and application areas of monitoring approaches proposed in different domains.Systematically identify frameworks supporting requirements monitoring.Analyze to what extent the monitoring frameworks support requirements monitoring in SoS. ContextSoftware systems today often interoperate with each other, thus forming a system of systems (SoS). Due to the scale, complexity, and heterogeneity of SoS, determining compliance with their requirements is challenging, despite the range of existing monitoring approaches. The fragmented research landscape and the diversity of existing approaches, however, make it hard to understand and analyze existing research regarding its suitability for SoS. ObjectiveThe aims of this paper are thus to systematically identify, describe, and classify existing approaches for requirements-based monitoring of software systems at runtime. Specifically, we (i) analyze the characteristics and application areas of monitoring approaches proposed in different domains, we (ii) systematically identify frameworks supporting requirements monitoring, and finally (iii) analyze their support for requirements monitoring in SoS. MethodWe performed a systematic literature review (SLR) to identify existing monitoring approaches and to classify their key characteristics and application areas. Based on this analysis we selected requirements monitoring frameworks, following a definition by Robinson, and analyzed them regarding their support for requirements monitoring in SoS. ResultsWe identified 330 publications, which we used to produce a comprehensive overview of the landscape of requirements monitoring approaches. We analyzed these publications regarding their support for Robinsons requirements monitoring layers, resulting in 37 identified frameworks. We investigated how well these frameworks support requirements monitoring in SoS. ConclusionsWe conclude that most existing approaches are restricted to certain kinds of checks, particular types of events and data, and mostly also limited to one particular architectural style and technology. This lack of flexibility makes their application in an SoS context difficult. Also, systematic and automated variability management is still missing. Regarding their evaluation, many existing frameworks focus on measuring the performance overhead, while only few frameworks have been assessed in cases studies with real-world systems.

A requirements monitoring model for systems of systems

Many software systems today can be characterized as systems of systems (SoS) comprising interrelated and heterogeneous systems developed by diverse teams over many years. Due to their scale, complexity, and heterogeneity engineers face significant challenges when determining the compliance of SoS with their requirements. Requirements monitoring approaches are a viable solution for checking system properties at runtime. However, existing approaches do not adequately consider the characteristics of SoS: different types of requirements exist at different levels and across different systems; requirements are maintained by different stakeholders; and systems are implemented using diverse technologies. This paper describes a three-dimensional requirements monitoring model (RMM) for SoS providing the following contributions: (i) our approach allows modeling the monitoring scopes of requirements with respect to the SoS architecture; (ii) it employs event models to abstract from different technologies and systems to be monitored; and (iii) it supports instantiating the RMM at runtime depending on the actual SoS configuration. To evaluate the feasibility of our approach we created a RMM for a real-world SoS from the automation software domain. We evaluated the model by instantiating it using an existing monitoring framework and a simulator running parts of this SoS. The results indicate that the model is sufficiently expressive to support monitoring SoS requirements of a directed SoS. It further facilitates diagnosis by discovering violations of requirements across different levels and systems in realistic monitoring scenarios.

Evolving systems of systems: industrial challenges and research perspectives

An increasing number of software systems today are systems of systems (SoS) comprising decentralized and heterogeneous systems with operational and managerial independence. The evolution of SoS is the rule and not the exception in practice due to frequently changing requirements, technologies, and markets. However, providing adequate support for the evolution of SoS is rather challenging as their behavior often emerges at runtime and is difficult to predict. Hence, SoS must be monitored during simulation and operation to ensure compliance with its requirements. In this position paper, we present challenges for SoS evolution from the domain of industrial automation. We discuss existing approaches supporting SoS evolution and derive research issues. We outline requirements for SoS evolution support and present key capabilities of a flexible monitoring and evolution infrastructure. We conclude with a discussion of research perspectives.