Ali Shahrokni

Quality Requirements in Industrial Practice—An Extended Interview Study at Eleven Companies

In order to create a successful software product and assure its quality, it is not enough to fulfill the functional requirements, it is also crucial to find the right balance among competing quality requirements (QR). An extended, previously piloted, interview study was performed to identify specific challenges associated with the selection, tradeoff, and management of QR in industrial practice. Data were collected through semistructured interviews with 11 product managers and 11 project leaders from 11 software companies. The contribution of this study is fourfold: First, it compares how QR are handled in two cases, companies working in business-to-business markets and companies that are working in business-to-consumer markets. These two are also compared in terms of impact on the handling of QR. Second, it compares the perceptions and priorities of QR by product and project management, respectively. Third, it includes an examination of the interdependencies among quality requirements perceived as most important by the practitioners. Fourth, it characterizes the selection and management of QR in downstream development activities.

Prioritization of quality requirements: State of practice in eleven companies

Requirements prioritization is recognized as an important but challenging activity in software product development. For a product to be successful, it is crucial to find the right balance among competing quality requirements. Although literature offers many methods for requirements prioritization, the research on prioritization of quality requirements is limited. This study identifies how quality requirements are prioritized in practice at 11 successful companies developing software intensive systems. We found that ad-hoc prioritization and priority grouping of requirements are the dominant methods for prioritizing quality requirements. The results also show that it is common to use customer input as criteria for prioritization but absence of any criteria was also common. The results suggests that quality requirements by default have a lower priority than functional requirements, and that they only get attention in the prioritizing process if decision-makers are dedicated to invest specific time and resources on QR prioritization. The results of this study may help future research on quality requirements to focus investigations on industry-relevant issues.

A systematic review of software robustness

Context: With the increased use of software for running key functions in modern society it is of utmost importance to understand software robustness and how to support it. Although there have been many contributions to the field there is a lack of a coherent and summary view. Objective: To address this issue, we have conducted a literature review in the field of robustness. Method: This review has been conducted by following guidelines for systematic literature reviews. Systematic reviews are used to find and classify all existing and available literature in a certain field. Results: From 9193 initial papers found in three well-known research databases, the 144 relevant papers were extracted through a multi-step filtering process with independent validation in each step. These papers were then further analyzed and categorized based on their development phase, domain, research, contribution and evaluation type. The results indicate that most existing results on software robustness focus on verification and validation of Commercial of the shelf (COTS) or operating systems or propose design solutions for robustness while there is a lack of results on how to elicit and specify robustness requirements. The research is typically solution proposals with little to no evaluation and when there is some evaluation it is primarily done with small, toy/academic example systems. Conclusion: We conclude that there is a need for more software robustness research on real-world, industrial systems and on software development phases other than testing and design, in particular on requirements engineering.

One-dimensional force feedback slider: going from an analogue to a digital platform

This paper examines the use of motorized physical sliders with position and force as input and output parameters for tangible human computer interaction. Firstly, we present an analogue platform. It was used to realize two proof-of-concept applications: one for learning system dynamics as part of physics education and the second for interaction with music loops. Based on the insight gained with the analogue platform and the two applications, we took the first steps towards a digital platform, also presented here. More generally, the paper presents so-called haptic modes, which may be generated using force feedback control of motorized sliders. The paper also presents parts of the underlying software and hardware which was designed and realized as part of this project.This paper examines the use of motorized physical sliders with position and force as input and output parameters for tangible human computer interaction. Firstly, we present an analogue platform. It was used to realize two proof-of-concept applications: one for learning system dynamics as part of physics education and the second for interaction with music loops. Based on the insight gained with the analogue platform and the two applications, we took the first steps towards a digital platform, also presented here. More generally, the paper presents so-called haptic modes, which may be generated using force feedback control of motorized sliders. The paper also presents parts of the underlying software and hardware which was designed and realized as part of this project.

BounceSlider: actuated sliders for music performance and composition

The ForceFeedback Slider (FFS) is a one-dimensional actuated slider using a motor to produce tangible interaction with position and force as input and output parameters. To create a new concept, we have built a mixing desk, placed six FFSs (two implemented here) into a partially realized SliderBox, and added a LED and two toggle buttons to each slider for additional interactivity. We have developed a tool called BounceSlider for improvising music. This application for real time music performance and composition uses a slider handle that can act as a ball. Users can lift and release the handle to set the ball in motion and produce a particular sound each time it bounces against the baseline. Based on physical characteristics, the user can create different sounds and loops by changing two settings: gravity (speed) and bounce type (ball physical characteristics). BounceSlider allows the user to create and save loops of Musical Instrument Digital Interface (Midi) with up to five sounds at a time.

Towards a Framework for Specifying Software Robustness Requirements Based on Patterns

[Context and motivation] With increasing use of software, quality attributes grow in relative importance. Robustness is a software quality attribute that has not received enough attention in requirements engineering even though it is essential, in particular for embedded and real-time systems. [Question/Problem] A lack of structured methods on how to specify robustness requirements generally has resulted in incomplete specification and verification of this attribute and thus potentially a lower quality. Currently, the quality of robustness specification is mainly dependent on stakeholder experience and varies wildly between companies and projects. [Principal idea/results] Methods targeting other non-functional properties such as safety and performance suggest that certain patterns occur in specification of requirements, regardless of project and company context. Our initial analysis with industrial partners suggests robustness requirements from different projects and contexts, if specified at all, follow the same rule. [Contribution] By identifying and gathering these commonalities into patterns we present a framework, ROAST, for specification of robustness requirements. ROAST gives clear guidelines on how to elicit and benchmark robustness requirements for software on different levels of abstraction.

RobusTest: A Framework for Automated Testing of Software Robustness

Robustness of a software system is defined as the degree to which the system can behave ordinarily and in conformance with the requirements in extraordinary situations. By increasing the robustness many failures which decrease the quality of the system can be avoided or masked. When it comes to specifying, testing and assessing software robustness in an efficient manner the methods and techniques are not mature yet. This paper presents RobusTest, a framework for testing robustness properties of a system with currently focus on timing issues. The expected robust behavior of the system is formulated as properties. The properties are then used to automatically generate robustness test cases and assess the results. An implementation of RobusTest in Java is presented here together with results from testing different, open-source implementations of the XMPP instant messaging protocol. By executing 400 test cases that were automatically generated from properties on two such implementations we found 11 critical failures and 15 nonconformance problems as compared to the XMPP specification.

Organic Evolution of Development Organizations – An Experience Report

In areas such as Active Safety, new technologies, designs (e.g. AUTOSAR) and methods are introduced at a rapid pace. To address the new demands, and also requirements on Functional Safety imposed by ISO 26262, the support for engineering methods, including tools and data management, needs to evolve as well. Generic and file-based data management tools, like spreadsheet tools, are popular in the industry due to their flexibility and legacy in the industry but provide poor control and traceability, while rigid and special-purpose tools provide structure and control of data but with limited evolvability. As organizations become agile, the need for flexible data management increases. Since products become more complex and developed in larger and distributed teams, the need for more unified, controlled, and consistent data increases. In this paper we report the successful experience of Volvo Car Corporation (VCC) to achieve these challenges and we present a new paradigm called organic evolution of development organization. VCC has for more than ten years used an integrated model-based platform for developing their active safety systems. The platform supports distributed real-time collaboration, fine-grained versioning, and integration with specialized tools. VCC uses the platform for requirements, design, test and verification, failure mode avoidance and change management. Specifications to suppliers, ISO 26262 safety documentation and AUTOSAR templates are generated from models. The agile and lean development processes have met the increased rate of evolution in projects like the new XC90. High quality is attested by both the products and the endorsement of the development organization.

Industrial Challenges with Quality Requirements in Safety Critical Software Systems

Budget constraints and the difficulty to specify quality requirements, such as reliability, robustness, and safety present challenges to many software companies in particular if they develop safety-critical systems. Failing to specify this type of requirements properly can lead to misunderstandings between the developers and the customers, which can threaten the quality of the system. However, little information is available on how companies currently work with these requirements. This paper describes the state of requirements engineering practice and identifies challenges with quality requirements by conducting a requirements analysis on two projects, including 980 requirements, at a company developing safety-critical systems. We also compare to similar results for a company developing end user software. Problematic and ambiguous requirements have caused misunderstandings with the customers and introduced late and project-critical costs to the projects. The results show that despite a high percentage of quality requirements at the company, the majority of these requirements are non-quantified, which potentially leads to lower verifiability, high ambiguity and can create late-project delays in verification and validation. Our study adds empirical evidence on the characteristics and challenges of real-world requirements engineering for safety-critical systems.

Traceability Data in Early Development Phases as an Enabler for Decision Support

Traceability information between requirements, architectural elements and the results of test cases can be used to unearth interesting relationships between the early phases of the software development process and the software faults in the end product. For instance, complex dependencies between features and software components could lead to an increased level of flaws in the code. Such patterns can be detected and visualized as early warnings to the relevant stakeholders (e.g., the architect or the project manager). Ultimately, a fully-fledged prediction model can be developed if enough historical information is available from previous software projects. In this paper we introduce a method for building a decision support system based on historic product data.