Krzysztof Wnuk

An Industrial Survey of Safety Evidence Change Impact Analysis Practice

Context. In many application domains, critical systems must comply with safety standards. This involves gathering safety evidence in the form of artefacts such as safety analyses, system specifications, and testing results. These artefacts can evolve during a systems lifecycle, creating a need for change impact analysis to guarantee that system safety and compliance are not jeopardised. Objective. We aim to provide new insights into how safety evidence change impact analysis is addressed in practice. The knowledge about this activity is limited despite the extensive research that has been conducted on change impact analysis and on safety evidence management. Method. We conducted an industrial survey on the circumstances under which safety evidence change impact analysis is addressed, the tool support used, and the challenges faced. Results. We obtained 97 valid responses representing 16 application domains, 28 countries, and 47 safety standards. The respondents had most often performed safety evidence change impact analysis during system development, from system specifications, and fully manually. No commercial change impact analysis tool was reported as used for all artefact types and insufficient tool support was the most frequent challenge. Conclusion. The results suggest that the different artefact types used as safety evidence co-evolve. In addition, the evolution of safety cases should probably be better managed, the level of automation in safety evidence change impact analysis is low, and the state of the practice can benefit from over 20 improvement areas.

Supporting Change Impact Analysis Using a Recommendation System: An Industrial Case Study in a Safety-Critical Context

Change Impact Analysis (CIA) during software evolution of safety-critical systems is a labor-intensive task. Several authors have proposed tool support for CIA, but very few tools were evaluated in industry. We present a case study on ImpRec, a recommendation System for Software Engineering (RSSE), tailored for CIA at a process automation company. ImpRec builds on assisted tracing, using information retrieval solutions and mining software repositories to recommend development artifacts, potentially impacted when resolving incoming issue reports. In contrast to the majority of tools for automated CIA, ImpRec explicitly targets development artifacts that are not source code. We evaluate ImpRec in a two-phase study. First, we measure the correctness of ImpRec’s recommendations by a simulation based on 12 years’ worth of issue reports in the company. Second, we assess the utility of working with ImpRec by deploying the RSSE in two development teams on different continents. The results suggest that ImpRec presents about 40 percent of the true impact among the top-10 recommendations. Furthermore, user log analysis indicates that ImpRec can support CIA in industry, and developers acknowledge the value of ImpRec in interviews. In conclusion, our findings show the potential of reusing traceability associated with developers’ past activities in an RSSE.

Open innovation using open source tools: a case study at Sony Mobile

Despite growing interest of Open Innovation (OI) in Software Engineering (SE), little is known about what triggers software organizations to adopt it and how this affects SE practices. OI can be realized in numerous of ways, including Open Source Software (OSS) involvement. Outcomes from OI are not restricted to product innovation but also include process innovation, e.g. improved SE practices and methods. This study explores the involvement of a software organization (Sony Mobile) in OSS communities from an OI perspective and what SE practices (requirements engineering and testing) have been adapted in relation to OI. It also highlights the innovative outcomes resulting from OI. An exploratory embedded case study investigates how Sony Mobile use and contribute to Jenkins and Gerrit; the two central OSS tools in their continuous integration tool chain. Quantitative analysis was performed on change log data from source code repositories in order to identify the top contributors and triangulated with the results from five semi-structured interviews to explore the nature of the commits. The findings of the case study include five major themes: i) The process of opening up towards the tool communities correlates in time with a general adoption of OSS in the organization. ii) Assets not seen as competitive advantage nor a source of revenue are made open to OSS communities, and gradually, the organization turns more open. iii) The requirements engineering process towards the community is informal and based on engagement. iv) The need for systematic and automated testing is still in its infancy, but the needs are identified. v) The innovation outcomes included free features and maintenance, and were believed to increase speed and quality in development. Adopting OI was a result of a paradigm shift of moving from Windows to Linux. This shift enabled Sony Mobile to utilize the Jenkins and Gerrit communities to make their internal development process better for its software developers and testers.

Practitioners’ Perspectives on Change Impact Analysis for Safety-Critical Software – A Preliminary Analysis

Safety standards prescribe change impact analysis (CIA) during evolution of safety-critical software systems. Although CIA is a fundamental activity, there is a lack of empirical studies about how it is performed in practice. We present a case study on CIA in the context of an evolving automation system, based on 14 interviews in Sweden and India. Our analysis suggests that engineers on average spend 50–100 h on CIA per year, but the effort varies considerably with the phases of projects. Also, the respondents presented different connotations to CIA and perceived the importance of CIA differently. We report the most pressing CIA challenges, and several ideas on how to support future CIA. However, we show that measuring the effect of such improvement solutions is non-trivial, as CIA is intertwined with other development activities. While this paper only reports preliminary results, our work contributes empirical insights into practical CIA.

Supporting Scope Tracking and Visualization for Very Large-Scale Requirements Engineering-Utilizing FSC+, Decision Patterns, and Atomic Decision Visualizations

Deciding the optimal project scope that fulfills the needs of the most important stakeholders is challenging due to a plethora of aspects that may impact decisions. Large companies that operate in rapidly changing environments experience frequently changing customer needs which force decision makers to continuously adjust the scope of their projects. Change intensity is further fueled by fierce market competition and hard time-to-market deadlines. Staying in control of the changes in thousands of features becomes a major issue as information overload hinders decision makers from rapidly extracting relevant information. This paper presents a visual technique, called Feature Survival Charts+ (FSC+), designed to give a quick and effective overview of the requirements scoping process for Very Large-Scale Requirements Engineering (VLSRE). FSC+ were applied at a large company with thousands of features in the database and supported the transition from plan-driven to a more dynamic and change-tolerant release scope management process. FSC+ provides multiple views, filtering, zooming, state-change intensity views, and support for variable time spans. Moreover, this paper introduces five decision archetypes deduced from the dataset and subsequently analyzed and the atomic decision visualization that shows the frequency of various decisions in the process. The capabilities and usefulness of FSC+, decision patterns (state changes that features undergo) and atomic decision visualizations are evaluated through interviews with practitioners who found utility in all techniques and indicated that their inherent flexibility was necessary to meet the varying needs of the stakeholders.

Supporting Strategic Decision-Making for Selection of Software Assets

Companies developing software are constantly striving to gain or keep their competitive advantage on the market. To do so, they should balance what to develop themselves and what to get from elsewhere, which may be software components or software services. These strategic decisions need to be aligned with business objectives and the capabilities and constraints of possible options. These sourcing options include: in-house, COTS, open source and outsourcing. The objective of this paper is to present an approach to support decision-makers in selecting appropriate types of origins in a specific case that maximizes the benefits of the selected business strategy. The approach consists of three descriptive models, as well as a decision process and a knowledge repository. The three models are a decision model that comprises three cornerstones (stakeholders, origins and criteria) and is based on a taxonomy for formulating decision models in this context, and two supporting models (property models and context models).

Choosing Component Origins for Software Intensive Systems: In-House, COTS, OSS or Outsourcing?—A Case Survey

The choice of which software component to use influences the success of a software system. Only a few empirical studies investigate how the choice of components is conducted in industrial practice. This is important to understand to tailor research solutions to the needs of the industry. Existing studies focus on the choice for off-the-shelf (OTS) components. It is, however, also important to understand the implications of the choice of alternative component sourcing options (CSOs), such as outsourcing versus the use of OTS. Previous research has shown that the choice has major implications on the development process as well as on the ability to evolve the system. The objective of this study is to explore how decision making took place in industry to choose among CSOs. Overall, 22 industrial cases have been studied through a case survey. The results show that the solutions specifically for CSO decisions are deterministic and based on optimization approaches. The non-deterministic solutions proposed for architectural group decision making appear to suit the CSO decision making in industry better. Interestingly, the final decision was perceived negatively in nine cases and positively in seven cases, while in the remaining cases it was perceived as neither positive nor negative.

A decision-making process-line for selection of software asset origins and components

Selecting sourcing options for software assets and components is an important process that helps companies to gain and keep their competitive advantage. The sourcing options include: in-house, COTS ...

Supporting Continuous Changes to Business Intents

Software supporting an enterprise’s business, also known as a business support system, needs to support the correlation of activities between actors as well as influence the activities based on knowledge about the value networks in which the enterprise acts. This requires the use of policies and rules to guide or enforce the execution of strategies or tactics within an enterprise as well as in collaborations between enterprises. With the help of policies and rules, an enterprise is able to capture an actor’s intent in its business support system, and act according to this intent on behalf of the actor. Since the value networks an enterprise is part of will change over time the business intents’ life cycle states might change. Achieving the changes in an effective and efficient way requires knowledge about the affected intents and the correlation between intents. The aim of the study is to identify how a business support system can support continuous changes to business intents. The first step is to find a theoretical model which serves as a foundation for intent-driven systems. We conducted a case study using a focus group approach with employees from Ericsson. This case study was influenced by the spiral case study process. The study resulted in a model supporting continuous definition and execution of an enterprise. The model is divided into three layers; Define, Execute, and a common governance view layer. This makes it possible to support continuous definition and execution of business intents and to identify the actors needed to support the business intents’ life cycles. This model is supported by a meta-model for capturing information into viewpoints. The research question is addressed by suggesting a solution supporting continuous definition and execution of an enterprise as a model of value architecture components and business functions. The results will affect how Ericsson will build the business studio for their next generation business support systems.

A systematic mapping study on requirements scoping

Context: Requirements scoping is one of the key activities in requirements management but also a major risk for project management. Continuously changing scope may create a congestion state in handling the requirements inflow which causes negative consequences, e.g. delays or scope creep. Objectives: In this paper, we look at requirements scoping literature outside Software Product Line (SPL) by exploring the current literature on the phenomenon, summarizing publication trends, performing thematic analysis and analyzing the strength of the evidence in the light of rigor and relevance assessment. Method: We run a Systematic Mapping Study (SMS) using snowballing procedure, supported by a database search for the start set identification, and identified 21 primary studies and 2 secondary studies. Results: The research interest in this area steadily increases and includes mainly case studies, validation or evaluation studies. The results were categorized into four themes: definitions, negative effects associated with scoping, challenges and identified methods/tools. The identified scope management techniques are also matched against the identified requirements scoping challenges.