Adam Czauderna

A machine learning approach for tracing regulatory codes to product specific requirements

Regulatory standards, designed to protect the safety, security, and privacy of the public, govern numerous areas of software intensive systems. Project personnel must therefore demonstrate that an as-built system meets all relevant regulatory codes. Current methods for demonstrating compliance rely either on after-the-fact audits, which can lead to significant refactoring when regulations are not met, or else require analysts to construct and use traceability matrices to demonstrate compliance. Manual tracing can be prohibitively time-consuming; however automated trace retrieval methods are not very effective due to the vocabulary mismatches that often occur between regulatory codes and product level requirements. This paper introduces and evaluates two machine-learning methods, designed to improve the quality of traces generated between regulatory codes and product level requirements. The first approach uses manually created traceability matrices to train a trace classifier, while the second approach uses web-mining techniques to reconstruct the original trace query. The techniques were evaluated against security regulations from the USA governments Health Insurance Privacy and Portability Act (HIPAA) traced against ten healthcare related requirements specifications. Results demonstrated improvements for the subset of HIPAA regulations that exhibited high fan-out behavior across the requirements datasets.

Towards mining replacement queries for hard-to-retrieve traces

Automated trace retrieval methods can significantly reduce the cost and effort needed to create and maintain requirements traces. However, the set of generated traces is generally quite imprecise and must be manually evaluated by analysts. In applied settings when the retrieval algorithm is unable to find the relevant links for a given query, a human user can improve the trace results by manually adding additional search terms and filtering out unhelpful ones. However, the effectiveness of this approach is largely dependent upon the knowledge of the user. In this paper we present an automated technique for replacing the original query with a new set of query terms. These query terms are learned through seeding a web-based search with the original query and then processing the results to identify a set of domain-specific terms. The query-mining algorithm was evaluated and fine-tuned using security regulations from the USA governments Health Insurance Privacy and Portability Act (HIPAA) traced against ten healthcare related requirements specifications.

TraceLab: an experimental workbench for equipping researchers to innovate, synthesize, and comparatively evaluate traceability solutions

TraceLab is designed to empower future traceability research, through facilitating innovation and creativity, increasing collaboration between researchers, decreasing the startup costs and effort of new traceability research projects, and fostering technology transfer. To this end, it provides an experimental environment in which researchers can design and execute experiments in TraceLabs visual modeling environment using a library of reusable and user-defined components. TraceLab fosters research competitions by allowing researchers or industrial sponsors to launch research contests intended to focus attention on compelling traceability challenges. Contests are centered around specific traceability tasks, performed on publicly available datasets, and are evaluated using standard metrics incorporated into reusable TraceLab components. TraceLab has been released in beta-test mode to researchers at seven universities, and will be publicly released via CoEST.org in the summer of 2012. Furthermore, by late 2012 TraceLabs source code will be released as open source software, licensed under GPL. TraceLab currently runs on Windows but is designed with cross platforming issues in mind to allow easy ports to Unix and Mac environments.

Grand challenges, benchmarks, and TraceLab: developing infrastructure for the software traceability research community

The challenges of implementing successful and cost-effective traceability have created a compelling research agenda that has addressed a broad range of traceability related issues, ranging from qualitative studies of traceability users in industry to very technical and quantitative studies. Unfortunately, advances are hampered by the significant time and effort needed to establish a traceability research environment and to perform comparative evaluations of new results against existing baselines. In this panel we discuss ongoing efforts by members of the Center of Excellence for Software Traceability (CoEST) to define the Grand Challenges of Traceability, develop benchmarks, and to construct TraceLab, an extensible and scalable visual environment for designing and executing a broad range of traceability experiments.

Traceability challenge 2011: using TraceLab to evaluate the impact of local versus global IDF on trace retrieval

Numerous trace retrieval algorithms incorporate the standard tf-idf (term frequency, inverse document frequency) technique to weight various terms. In this paper we address Grand Challenge C-GC1 by comparing the effectiveness of computing idf based only on the local terms in the query, versus computing it based on general term usage as documented in the American National Corpus. We also address Grand Challenges L-GC1 and L-GC2 by setting ourselves the additional task of designing and conducting the experiments using the alpha-release of TraceLab. TraceLab is an experimental workbench which allows researchers to graphically model and execute a traceability experiment as a workflow of components. Results of the experiment show that the local idf approach exceeds or matches the global approach in all of the cases studied.

Chapter 4 – Driving Architectural Design and Preservation from a Persona Perspective in Agile Projects

Architecturally significant requirements (ASRs) drive and constrain many aspects of architecture. Eliciting and analyzing these requirements in the early phases of a project means that quality concerns can be discovered and addressed during the architectural design. This reduces the risk of costly and unnecessary refactoring. The challenge of emerging requirements is particularly evident in agile projects, which are inherently incremental; however, existing techniques for eliciting ASRs, such as win-win and i*, are typically rejected by agile development teams as being somewhat heavyweight. In this chapter, we present the notion of an architecturally savvy persona (ASP), which is used to emerge and analyze stakeholders’ quality concerns and to drive and validate the architectural design. ASPs are useful for discovering, analyzing, and managing ASRs, and designing and validating high-level architectural solutions that balance tradeoffs and satisfy stakeholders’ concerns. We show how ASPs can be used to discover quality concerns, drive architectural design, and preserve architectural qualities during long-term maintenance activities.

Just-in-time traceability for mechatronics systems

Effective traceability can be very costly and difficult to achieve in mechatronics systems due to their overall size and complexity. Such systems are often specified and designed in terms of software, electrical, mechanical, and thermodynamic elements, and associated models are represented and stored in various formats and locations. Traceability is of critical concern in such systems, which must often demonstrably comply to a wide range of regulatory codes. To address this problem we present an enterprise level architectural solution for establishing organization wide traceability across a variety of model types. While our approach is applicable to any enterprise level tracing environment, we focus on the mechatronics problem and illustrate our solution with a prototype tool that delivers semi-automated traceability between the regulatory codes, requirements, and systems models of a mechatronics system.

Using tracelab to design, execute, and baseline empirical requirements engineering experiments

As Requirements Engineering research continues to grow into a mature and rigorous discipline, an increasing focus is placed on the need for sound evaluation techniques that compare the benefits of a new solution against existing ones. In this tool demonstration we introduce TraceLab, an instrumented environment for modeling, executing, and comparatively evaluating experimental results. While initially developed for the Software Traceability domain, TraceLab provides a framework which can be populated with experiments, datasets, and reusable components for almost any empirical software engineering domain. In this demo we present examples from the Requirements Engineering domain.