Tom-Michael Hesse

A Platform for Empirical Research on Information System Evolution

Software-intensive systems are subject to continuous change due to modification of the systems themselves and their environment. Methods for supporting evolution are a competitive edge in software engineering as software is operated over decades. Empirical research is useful to validate the effectiveness of these methods. However, empirical studies on software evolution are rarely comprehensive and hardly replicable. Collaboration in empirical studies may prevent these shortcomings. We analyzed the support for such collaboration and examined existing studies in a literature review. Based on our findings, we designed CoCoMEP‐ a platform for supporting collaboration in empirical research on software evolution by shared knowledge. We report lessons learned from the application of the platform in a large research programme.

Supporting the collaborative development of requirements and architecture documentation

In most software projects, particular requirements significantly drive the design of the software architecture by forcing architectural decisions to be made. As requirements and architecture are refined iteratively, their extensions and improvements need to be aligned continuously. Much research has been conducted to identify such requirements and their impact on architecture. However, it remains a problem how to collaboratively document such requirements and architectural knowledge under development. In particular, knowledge of architectural decisions such as assumptions or alternatives for the system erodes over time and can even vaporize completely. A major reason is the inability to easily manage informality and complexity of knowledge when performing both requirements engineering and architecture design. Therefore, we propose a documentation model for decisions supporting the intertwined documentation of related requirements and architecture knowledge. It provides documentation elements, which are common to both disciplines. In order to support refinement in documentation, knowledge can be iteratively accumulated at different levels of granularity. So the model fits to the twin peaks model of requirements and architecture. In consequence, the comprehension and collaboration between requirements engineers and system architects is improved by negotiating and refining the same documentation together in an ongoing process. We apply our approach to an example in order to demonstrate that it is applicable and useful for managing architectural decision knowledge in relation to the grounding requirements.

Documenting Implementation Decisions with Code Annotations

Software developers make various decisions when implementing software. For instance, they decide on how to implement an algorithm most efficiently or in which way to process user input. When code is revisited during maintenance, the underlying decisions need to be understood and possibly adjusted to the current situation. Common documentation approaches like JavaDoc neither cover knowledge related to decisions explicitly, nor are they integrated closely with knowledge management. In consequence, decision knowledge is rarely documented and therefore inaccessible, especially when developers have left the team. So, effective maintenance is hindered. We have developed an annotation model for decision knowledge and integrated it with the knowledge management tool UNICASE. The approach enables developers to document decisions within code without tool switches to lower their documentation effort. Afterwards, maintainers can exploit the embedded decision knowledge and follow links to external knowledge. This paper presents the approach and evaluation results of a first case study, which indicate its practicability.

Semiautomatic security requirements engineering and evolution using decision documentation, heuristics, and user monitoring

Security issues can have a significant negative impact on the business or reputation of an organization. In most cases they are not identified in requirements and are not continuously monitored during software evolution. Therefore, the inability of a system to conform to regulations or its endangerment by new vulnerabilities is not recognized. In consequence, decisions related to security might not be taken at all or become obsolete quickly. But to evaluate efficiently whether an issue is already addressed appropriately, software engineers need explicit decision documentation. Often, such documentation is not performed due to high overhead. To cope with this problem, we propose to document decisions made to address security requirements. To lower the manual effort, information from heuristic analysis and end user monitoring is incorporated. The heuristic assessment method is used to identify security issues in given requirements automatically. This helps to uncover security decisions needed to mitigate those issues. We describe how the corresponding security knowledge for each issue can be incorporated into the decision documentation semiautomatically. In addition, violations of security requirements at runtime are monitored. We show how decisions related to those security requirements can be identified through the documentation and updated manually. Overall, our approach improves the quality and completeness of security decision documentation to support the engineering and evolution of security requirements.

Documenting Relations Between Requirements and Design Decisions: A Case Study on Design Session Transcripts

Context/Motivation: Developers make many important decisions as they address given requirements during system design. Each decision is explained and justified by decision-related knowledge. Typically, this knowledge is neither captured in a structured way, nor linked to the respective requirements in detail. Then, it is not obvious, how design decisions realize the given requirements and whether they further refine or shape them. Thus, the relations and alignment of requirements and design cannot be assessed properly. Problem/Question: While there are several studies on decision-making in general, there does not exist a study uncovering how decision-related knowledge emerges based on requirements. Such a study is important to understand the intertwined relations of requirements and design decisions as well as how requirement descriptions could be enhanced with feedback from design decision-making. Principal Idea/Results: We applied a flexible documentation approach for decision-related knowledge on discussion transcripts of two design sessions with professional designers. We analyzed the discussions for decision-related knowledge and documented it together with its relations to the given requirements. Several complex and incrementally growing knowledge structures for decisions were found to emerge in relation to the given requirements. Also, we uncovered that decision-related knowledge contained uncertainties about requirements and further refined them. Contribution: Our study uncovers detailed relations between requirements and design decisions and thereby improves the understanding of their mutual impact on each other. We also derive recommendations for the cooperation between requirements engineers and designers in practice. In addition, we demonstrate that our documentation approach for decision-related knowledge provides a comprehensive view on decisions and their relations to requirements.

DecDoc: A Tool for Documenting Design Decisions Collaboratively and Incrementally

The outcome and quality of design decisions highly depend on the knowledge reflected during decision-making. Typically, making design decisions is not one singular action.Instead, developers discuss and cooperate during requirements engineering, design and implementation of a system to make and adapt design decisions. This decision-making process is influenced by different decision-making strategies, personal experiences, and biases. In consequence, decision-related knowledge emerges incrementally over time in an incomplete and heterogeneous way. This hinders the documentation of such knowledge in practice. First, most documentation tools capture decision-related knowledge within one particular development activity.However, they do not focus on the collaborative and shared documentation during multiple activities. Second, static documentation templates and formal rules are not suitable for capturing incomplete knowledge, as additional documentation effort is imposed for developers. Thus, text templates are not used or filled with generic contents. As a result, decision-related knowledge remains implicit and is not available to guide future decision-making. To address these issues, we have created the tool DecDoc based on our incremental documentation model. The tool enables developers to capture decision-related knowledge and collaborate on a comprehensive documentation of design decisions with relations to artifacts, such as requirement specifications, design diagrams, and code. This helps to improve the decision-making process for design decisions, as it helps to make explicit and reflect related knowledge during the process. In this paper, we present DecDoc with regard to requirements from the decision-making process.Then, we describe its application on design decisions in example projects. Finally, we discuss our insights from using the tool and highlight open challenges.

Supporting Project Management Through Integrated Management of System and Project Knowledge

Software engineering is a knowledge-intensive task. Many different kinds of knowledge are created, for example, system knowledge, such as requirements, design, or code, and project knowledge, such as project plans, decisions, and work items. In this chapter, we study two kinds of project knowledge: work items and decisions. Work items document what should or has been done by whom and when. Decisions represent the solution to a decision problem. They are important to be kept in mind so that the future development will be consistent with the past. These kinds of knowledge can be implicit or explicit. Work items are typically managed explicitly in issue trackers, while decisions are mostly hidden in informal notes or in the artifacts, which result from these decisions. Rationale management research has suggested several approaches to make decisions and their rationale explicit. However, in contrast to issue trackers, which are widespread, these approaches are considered as overhead in industry. In this chapter, we argue that work items and decisions should be managed together with the system knowledge. This has several benefits for project management processes, such as project planning or monitoring, for example, with better information for the allocation of work items or risk identification. We present a vision detailing these benefits and discuss what is known in research and practice about the realization of this vision. In particular, we review existing approaches to capture work items or decisions and their links to other knowledge and discuss the empirical evidence of their benefits for an integrated system and project knowledge management in industry.

Documented decision-making strategies and decision knowledge in open source projects

Empirical study on documented decisions based on 260 Firefox issue discussions.Shows that more naturalistic than rational decision-making was documented.Shows that developers mostly documented knowledge on decision context and solution.Further research required on the impact and support of naturalistic documentation.Our coding tables can be used by other researchers for documentation analysis. Context: Decision-making is a vital task during software development. Typically, issue tracking systems are used to document decisions in large open source projects where developers are spread across the world. While most decision documentation approaches assume that developers use rational decision strategies, in practice also naturalistic strategies are employed. However, quantitative studies of the distribution of decision strategies and related knowledge are missing.Objective: Our overall goal is to provide insights and ideas for further research to systematically support and document decision-making during software development in open source projects. In this paper, we analyze decisions documented in comments to issue reports in order to understand the documentation of decision-making in detail.Method: We coded the comments of 260 issue reports of the open source project Firefox for decision-making strategies and knowledge on decisions. Then, we statistically analyzed the coded data with regard to the dominant decision strategy, the distribution of decision strategies and knowledge, and the relations between strategy and knowledge.Results: The vast majority of documented decision-making strategies was naturalistic. Interestingly, for feature requests the percentage of rational decision-making strategies was higher than for bugs. Documented knowledge mostly concerned the decision context. More solutions were documented together with a higher amount of naturalistic decision-making. However, solutions were negatively correlated with the assessment of the situation. So, developers are likely to exploit and document decision problems and solutions in an imbalanced way.Conclusion: Our analysis revealed important insights on how decision-making and its related knowledge is documented during software development in open source projects. For instance, we found naturalistic decision-making to play an important role for development decisions. Our coding tables can be used by other researchers to further investigate our results. The study insights should be reflected in decision support systems to improve their effectiveness and acceptance by developers.

Human Aspects in Software Architecture Decision Making: A Literature Review

Despite past efforts, we have little understanding and limited research efforts on how architects make decisions in the real-world settings. It seems that software architecture researchers make implicit assumption that decision making by software architects can be a rational and prescribed process. Such an assumption is disputed in other fields such as economics and decision research. This paper studies the current state of software architecture decision making research in terms of human behaviors and practice. We carried out a literature review on software architecture decision making. We classified papers into decision making behavior and decision making practice and identified the research relationships between them. We found that decision making is a mental activity. Research into the behavioral aspects of software architecture decision making for incorporation into architectural design practices is required. We suggest three research topics on human aspects to improve software architecture practices.

Towards Identification of Software Improvements and Specification Updates by Comparing Monitored and Specified End-User Behavior

Support of end-user needs is an important success factor for a software application. In order to optimize the support of end-user needs, developers have to be aware of them and their evolution over time. But a communication gap between developers and users leads to ignorance of developers about how users use their application. Also, developer assumptions about user behavior are rarely tested and corrected if they are wrong. Consequently, many software applications have a mediocre support of user needs and user problems as well as changes in user needs are detected rather late. In this paper, we present a research agenda addressing this problem by comparing use case descriptions to monitored user actions. More specifically, we propose to monitor user actions using instrumentation, detect the current use case of a user using machine learning, and compare use case steps to monitored user actions. By detecting differences between both, we identify mismatches between user behavior and developer assumptions reflected in use case descriptions. Those mismatches can serve as starting points to identify software improvements, to test the use case specification and identify updates, and to revise training programs. Finally, we sketch a plan to evaluate our approach.