Grischa Liebel

Assessing the State-of-Practice of Model-Based Engineering in the Embedded Systems Domain

Model-Based Engineering (MBE) aims at increasing the effectiveness of engineering by using models as key artifacts in the development process. While empirical studies on the use and the effects of MBE in industry exist, there is only little work targeting the embedded systems domain. We contribute to the body of knowledge with a study on the use and the assessment of MBE in that particular domain. We collected quantitative data from 112 subjects, mostly professionals working with MBE, with the goal to assess the current State of Practice and the challenges the embedded systems domain is facing. Our main findings are that MBE is used by a majority of all participants in the embedded systems domain, mainly for simulation, code generation, and documentation. Reported positive effects of MBE are higher quality and improved reusability. Main shortcomings are interoperability difficulties between MBE tools, high training effort for developers and usability issues.

Model-based engineering in the embedded systems domain: an industrial survey on the state-of-practice

Model-based engineering (MBE) aims at increasing the effectiveness of engineering by using models as important artifacts in the development process. While empirical studies on the use and the effects of MBE in industry exist, only few of them target the embedded systems domain. We contribute to the body of knowledge with an empirical study on the use and the assessment of MBE in that particular domain. The goal of this study is to assess the current state-of-practice and the challenges the embedded systems domain is facing due to shortcomings with MBE. We collected quantitative data from 113 subjects, mostly professionals working with MBE, using an online survey. The collected data spans different aspects of MBE, such as the used modeling languages, tools, notations, effects of MBE introduction, or shortcomings of MBE. Our main findings are that MBE is used by a majority of all participants in the embedded systems domain, mainly for simulation, code generation, and documentation. Reported positive effects of MBE are higher quality and improved reusability. Main shortcomings are interoperability difficulties between MBE tools, high training effort for developers and usability issues. Our study offers valuable insights into the current industrial practice and can guide future research in the fields of systems modeling and embedded systems.

Requirements Engineering Challenges in Large-Scale Agile System Development

Motivated by their success in software development, companies implement agile methods and their practices increasingly for software-intense, large products, such as cars, telecommunication infrastructure, and embedded systems. Such systems are usually subject to safety and regulative concerns as well as different development cycles of hardware and software. Consequently, requirements engineering involves upfront and detailed analysis, which can be at odds with agile (software) development. In this paper, we present results from a multiple case study with two car manufacturers, a telecommunications company, and a technology company that are on the journey to introduce organization wide continuous integration and continuous delivery to customers. Based on 20 qualitative interviews, 5 focus groups, and 2 cross-company workshops, we discuss possible scopes of agile methods within system development, the consequences this has on the role of requirements, and the challenges that arise from the interplay of requirements engineering and agile methods in large-scale system development. These relate in particular to communicating and managing knowledge about a) customer value and b) the system under development. We conclude that better alignment of a holistic requirements model with agile development practices promises rich gains in development speed, flexibility, and overall quality of software and systems.

Quality Requirements in Agile as a Knowledge Management Problem: More than Just-in-Time

Just-in-time (JIT) approaches have been suggested for managing non-functional requirements in agile projects. However, many non-functional requirements cannot be raised and met on the spot. In this position paper, we argue that effective JIT engineering of quality requirements depends on a solid foundation of long-term knowledge about all relevant quality requirements. We present two examples from projects related to safety and security and show that not all aspects of these quality requirements can be invented and changed just in time. Further, managing, for example, operationalization of quality requirements just in time depends on sufficient understanding of (i) customer value and (ii) the system under construction that must be shared by the engineering team. If a Learning Software Organization (LSO) intends to increase agility and speed up system development, it needs a holistic concept for managing this knowledge. We propose that a knowledge-management framework can facilitate JIT-RE by structuring, representing, and allowing updates of long-term knowledge about quality requirements. Such a knowledge-management framework should allow to map user value to system requirements and have important properties to allow JIT RE and sustainable evolution.

Organisation and communication problems in automotive requirements engineering

Project success in the automotive industry is highly influenced by requirements engineering (RE), for which communication and organisation structure play a major role, much due to the scale and distribution of these projects. However, empirical research is scarce on these aspects of automotive RE and warrants closer examination. Therefore, the purpose of this paper is to identify problems or challenges in automotive RE with respect to communication and organisation structure. Using a multiple-case study approach, we collected data via 14 semi-structured interviews at one car manufacturer and one supplier. We tested our findings from the case study with a questionnaire distributed to practitioners in the automotive industry. Our results indicate that it is difficult but increasingly important to establish communication channels outside the fixed organisation structure and that responsibilities are often unclear. Product knowledge during early requirements elicitation and context knowledge later on is lacking. Furthermore, abstraction gaps between requirements on different abstraction levels leads to inconsistencies. For academia, we formulate a concrete agenda for future research. Practitioners can use the findings to broaden their understanding of how the problems manifest and to improve their organisations.

Verdict machinery: on the need to automatically make sense of test results

Along with technological developments and increasing competition there is a major incentive for companies to produce and market high quality products before their competitors. In order to conquer a bigger portion of the market share, companies have to ensure the quality of the product in a shorter time frame. To accomplish this task companies try to automate their test processes as much as possible. It is critical to investigate and understand the problems that occur during different stages of test automation processes. In this paper we report on a case study on automatic analysis of non-functional test results. We discuss challenges in the face of continuous integration and deployment and provide improvement suggestions based on interviews at a large company in Sweden. The key contributions of this work are filling the knowledge gap in research about performance regression test analysis automation and providing warning signs and a road map for the industry.

Challenges of Aligning Requirements Engineering and System Testing in Large-Scale Agile: A Multiple Case Study

As agile methods become more pervasive, agile practices are applied to more large-scale systems with a scope that goes beyond pure software. The expansion of agile in these contexts provides benefits, but creates new challenges. Widespread use of agile has changed the way we must think about practices both in Requirements Engineering (RE) and in System Testing (ST). Our experience shows that many challenges in the application of large-scale agile development relate to either RE or ST, and in particular to the alignment between these areas. In this paper we present large-scale agile-related challenges from a multiple case study which relate to REST alignment. We map our challenges to an existing framework for REST alignment, and make an initial attempt to suggest agile RE practices from the literature which may alleviate these challenges. Our results show that the interviewed companies need to first adopt more agile RE practices to enhance REST alignment and then leverage agile testing. Future work will look more towards evaluating these best practices.

Model-driven software engineering in the openETCS project: project experiences and lessons learned

Model-driven software engineering in industrial practice has been the focus of different empirical studies and experience reports. Particularly, positive effects of model-driven software engineering have been reported in the domain of embedded and safety-critical systems. We report in this paper on the experiences of the openETCS European research project whose goal was to formalize the System Requirements Specification and to develop an open source reference implementation of the European Train Control System including open source modeling tools. Furthermore, we will discuss lessons learned, e.g., about using open source modeling toolchains in safety-critical contexts and about using the SCADE Suite for the development of the safety-critical parts.

Use, potential, and showstoppers of models in automotive requirements engineering

Several studies report that the use of model-centric methods in the automotive domain is widespread and offers several benefits. However, existing work indicates that few modelling frameworks explicitly include requirements engineering (RE), and that natural language descriptions are still the status quo in RE. Therefore, we aim to increase the understanding of current and potential future use of models in RE, with respect to the automotive domain. In this paper, we report our findings from a multiple-case study with two automotive companies, collecting interview data from 14 practitioners. Our results show that models are used for a variety of different purposes during RE in the automotive domain, e.g. to improve communication and to handle complexity. However, these models are often used in an unsystematic fashion and restricted to few experts. A more widespread use of models is prevented by various challenges, most of which align with existing work on model use in a general sense. Furthermore, our results indicate that there are many potential benefits associated with future use of models during RE. Interestingly, existing research does not align well with several of the proposed use cases, e.g. restricting the use of models to informal notations for communication purposes. Based on our findings, we recommend a stronger focus on informal modelling and on using models for multi-disciplinary environments. Additionally, we see the need for future work in the area of model use, i.e. information extraction from models by non-expert modellers.

Involving External Stakeholders in Project Courses

Problem: The involvement of external stakeholders in capstone projects and project courses is desirable due to its potential positive effects on the students. Capstone projects particularly profit from the inclusion of an industrial partner to make the project relevant and help students acquire professional skills. In addition, an increasing push towards education that is aligned with industry and incorporates industrial partners can be observed. However, the involvement of external stakeholders in teaching moments can create friction and could, in the worst case, lead to frustration of all involved parties. Contribution: We developed a model that allows analysing the involvement of external stakeholders in university courses both in a retrospective fashion, to gain insights from past course instances, and in a constructive fashion, to plan the involvement of external stakeholders. Key Concepts: The conceptual model and the accompanying guideline guide the teachers in their analysis of stakeholder involvement. The model is comprised of several activities (define, execute, and evaluate the collaboration). The guideline provides questions that the teachers should answer for each of these activities. In the constructive use, the model allows teachers to define an action plan based on an analysis of potential stakeholders and the pedagogical objectives. In the retrospective use, the model allows teachers to identify issues that appeared during the project and their underlying causes. Drawing from ideas of the reflective practitioner, the model contains an emphasis on reflection and interpretation of the observations made by the teacher and other groups involved in the courses. Key Lessons: Applying the model retrospectively to a total of eight courses shows that it is possible to reveal hitherto implicit risks and assumptions and to gain a better insight into the interaction between external stakeholders and students. Our empirical data reveals seven recurring risk themes that categorise the different risks appearing in the analysed courses. These themes can also be used to categorise mitigation strategies to address these risks proactively. Additionally, aspects not related to external stakeholders, e.g., about the interaction of the project with other courses in the study programme, have been revealed. The constructive use of the model for one course has proved helpful in identifying action alternatives and finally deciding to not include external stakeholders in the project due to the perceived cost-benefit-ratio. Implications to Practice: Our evaluation shows that the model is a viable and useful tool that allows teachers to reason about and plan the involvement of external stakeholders in a variety of course settings, and in particular in capstone projects.