Patrick Rempel

Mind the gap: assessing the conformance of software traceability to relevant guidelines

Many guidelines for safety-critical industries such as aeronautics, medical devices, and railway communications, specify that traceability must be used to demonstrate that a rigorous process has been followed and to provide evidence that the system is safe for use. In practice, there is a gap between what is prescribed by guidelines and what is implemented in practice, making it difficult for organizations and certifiers to fully evaluate the safety of the software system. In this paper we present an approach, which parses a guideline to extract a Traceability Model depicting software artifact types and their prescribed traces. It then analyzes the traceability data within a project to identify areas of traceability failure. Missing traceability paths, redundant and/or inconsistent data, and other problems are highlighted. We used our approach to evaluate the traceability of seven safety-critical software systems and found that none of the evaluated projects contained traceability that fully conformed to its relevant guidelines.

An empirical study on project-specific traceability strategies

Effective requirements traceability supports practitioners in reaching higher project maturity and better product quality. Researchers argue that effective traceability barely happens by chance or through ad-hoc efforts and that traceability should be explicitly defined upfront. However, in a previous study we found that practitioners rarely follow explicit traceability strategies. We were interested in the reason for this discrepancy. Are practitioners able to reach effective traceability without an explicit definition? More specifically, how suitable is requirements traceability that is not strategically planned in supporting a projects development process. Our interview study involved practitioners from 17 companies. These practitioners were familiar with the development process, the existing traceability and the goals of the project they reported about. For each project, we first modeled a traceability strategy based on the gathered information. Second, we examined and modeled the applied software engineering processes of each project. Thereby, we focused on executed tasks, involved actors, and pursued goals. Finally, we analyzed the quality and suitability of a projects traceability strategy. We report common problems across the analyzed traceability strategies and their possible causes. The overall quality and mismatch of analyzed traceability suggests that an upfront-defined traceability strategy is indeed required. Furthermore, we show that the decision for or against traceability relations between artifacts requires a detailed understanding of the projects engineering process and goals; emphasizing the need for a goal-oriented procedure to assess existing and define new traceability strategies.

Requirements traceability across organizational boundaries: a survey and taxonomy

[Context and motivation] Outsourcing of software development is an attractive business model. Companies expect cost reduction, enhanced efficiency, and exploited external resources. However, this paradigmatic shift also introduces challenges as stakeholders are spread across distinct organizations. [Question/problem] Requirements traceability supports stakeholders in satisfying information needs about developments and could be a viable way of addressing the challenges of interorganizational development. While requirements traceability has been the subject of significant research efforts, its application across organizational boundaries is a largely unexplored area. [Principal ideas/results] We followed a qualitative research approach. First, we developed a taxonomy identifying the needs of inter-organizational traceability. Second, we conducted semi-structured interviews with informants from 17 companies. Eventually, we applied qualitative content analysis to extract findings that supported and evolved our taxonomy. [Contribution] Practitioners planning and managing inter-organizational relationships can use our findings as a conceptual baseline to effectively leverage traceability in those settings. Effective traceability supports projects in accomplishing their primary goal of maximizing business value.

Preventing Defects: The Impact of Requirements Traceability Completeness on Software Quality

Requirements traceability has long been recognized as an important quality of a well-engineered system. Among stakeholders, traceability is often unpopular due to the unclear benefits. In fact, little evidence exists regarding the expected traceability benefits. There is a need for empirical work that studies the effect of traceability. In this paper, we focus on the four main requirements implementation supporting activities that utilize traceability. For each activity, we propose generalized traceability completeness measures. In a defined process, we selected 24 medium to large-scale open-source projects. For each software project, we quantified the degree to which a studied development activity was enabled by existing traceability with the proposed measures. We analyzed that data in a multi-level Poisson regression analysis. We found that the degree of traceability completeness for three of the studied activities significantly affects software quality, which we quantified as defect rate. Our results provide for the first time empirical evidence that more complete traceability decreases the expected defect rate in the developed software. The strong impact of traceability completeness on the defect rate suggests that traceability is of great practical value for any kind of software development project, even if traceability is not mandated by a standard or regulation.

A quality model for the systematic assessment of requirements traceability

Traceability is an important quality of software requirements and allows to describe and follow their life throughout a development project. The importance of traceable requirements is reflected by the fact that requirements standards, safety regulations, and maturity models explicitly demand for it. In practice, traceability is created and maintained by humans, which make mistakes. In result, existing traces are potentially of dubious quality but serve as the foundation for high impact development decisions. We found in previous studies that practitioners miss clear guidance on how to systematically assess the quality of existing traces. In this paper, we review the elements involved in establishing traceability in a development project and derive a quality model that specifies per element the acceptable state (Traceability Gate) and unacceptable deviations (Traceability Problem) from this state. We describe and formally define how both, the acceptable states and the unacceptable deviations can be detected in order to enable practitioners to systematically assess their projects traceability. We evaluated the proposed model through an expert survey. The participating experts considered the quality model to be complete and attested that its quality criteria are of high relevance. We further found that the experts weight the occurrence of different traceability problems with different criticality. This information can be used to quantify the impact of traceability problems and to prioritize the assessment of traceability elements.

Estimating the Implementation Risk of Requirements in Agile Software Development Projects with Traceability Metrics

[Context and Motivation] Agile developments follow an iterative procedure with alternating requirements planning and implementation phases boxed into sprints. For every sprint, requirements from the product backlog are selected and appropriate test measures are chosen. [Question/problem] Both activities should carefully consider the implementation risk of each requirement. In favor of a successful project, risky requirements should either be deferred or extra test effort should be dedicated on them. Currently, estimating the implementation risk of requirements is mainly based on gut decisions. [Principal ideas/results] The complexity of the graph spanned by dependency and decomposition relations across requirements can be an indicator of implementation risk. In this paper, we propose three metrics to assess and quantify requirement relations. We conducted a study with five industry-scale agile projects and found that the proposed metrics are in fact suitable for estimating implementation risk of requirements. [Contribution] Our study of heterogeneous, industrial development projects delivers for the first time evidence that the complexity of a requirements traceability graph is correlated with the error-proneness of the implementing source code. The proposed traceability metrics provide an indicator for requirements’ implementation risks. This indicator supports product owners and developers in requirement prioritization and test measure selection.

Towards feature-aware retrieval of refinement traces

Requirements traceability supports practitioners in reaching higher project maturity and better product quality. To gain this support, traces between various artifacts of the software development process are required. Depending on the number of existing artifacts, establishing traces can be a time-consuming and error-prone task. Additionally, the manual creation of traces frequently interrupts the software development process. In order to overcome those problems, practitioners are asking for techniques that support the creation of traces (see Grand Challenge: Ubiquitous (GC-U)). In this paper, we propose the usage of a graph clustering algorithm to support the retrieval of refinement traces. Refinement traces are traces that exist between artifacts created in different phases of a development project, e.g., between features and use cases. We assessed the effectiveness of our approach in several TraceLab experiments. These experiments employ three standard datasets containing differing types of refinement traces. Results show that graph clustering can improve the retrieval of refinement traces and is a step towards the overall goal of ubiquitous traceability.

Recommending Auto-completions for Software Modeling Activities

Auto-completion of textual inputs benefits software developers using IDEs. However, graphical modeling tools used to design software do not provide this functionality. The challenges of recommending auto-completions for graphical modeling activities are largely unexplored. Recommending such auto-completions requires detecting meaningful partly completed activities, tolerating variance in user actions, and determining most relevant activities that a user wants to perform. This paper proposes an approach that works in the background while a developer is creating or evolving models and handles all these challenges. Editing operations are analyzed and matched to a predefined but extensible catalog of common modeling activities for structural UML models. In this paper we solely focus on determining recommendations rather than automatically completing activities. We demonstrated the quality of recommendations generated by our approach in a controlled experiment with 16 students evolving models.We recommended 88% of a users activities within a short list of ten recommendations.

The IlmSeven Dataset

Developing new ideas and algorithms or comparing new findings in the field of requirements engineering and management implies a dataset to work with. Collecting the required data is time consuming, tedious, and may involve unforeseen difficulties. The need for datasets often forces re-searchers to collect data themselves in order to evaluate their findings. However, comparing results with other publications is especially difficult on proprietary datasets. A big obstacle is the reproduction of a previously used dataset, which may include subtle preprocessing steps not explicitly mentioned by the original authors. Providing a predefined dataset avoids these problems. It establishes a common baseline and enables direct comparison for benchmarking. This paper provides a well defined dataset consisting of seven open source software projects. It contains a large number of typed development artifacts and links between them. Enriched with additional metadata, such as time stamps, versions, and component information, the dataset allows answering a broad range of research questions.

Continuous assessment of software traceability

Many guidelines for safety-critical industries, such as aeronautics, medical devices, and railway communications, specify that traceability must be used to demonstrate that a rigorous development process has been followed and to provide evidence that the developed system is safe for use. However, creating accurate and complete traceability is costly and remains a practical challenge. The significant cost and effort of the certification process makes it difficult to introduce changes once the product is certified. We propose an automated traceability assessment approach TRUST that can be used to assess software traceability in a continuous manner to ease the change process of certified products.