David Benavides

Automated analysis of feature models 20 years later: A literature review

Software product line engineering is about producing a set of related products that share more commonalities than variabilities. Feature models are widely used for variability and commonality management in software product lines. Feature models are information models where a set of products are represented as a set of features in a single model. The automated analysis of feature models deals with the computer-aided extraction of information from feature models. The literature on this topic has contributed with a set of operations, techniques, tools and empirical results which have not been surveyed until now. This paper provides a comprehensive literature review on the automated analysis of feature models 20 years after of their invention. This paper contributes by bringing together previously disparate streams of work to help shed light on this thriving area. We also present a conceptual framework to understand the different proposals as well as categorise future contributions. We finally discuss the different studies and propose some challenges to be faced in the future.

Automated reasoning on feature models

Software Product Line (SPL) Engineering has proved to be an effective method for software production. However, in the SPL community it is well recognized that variability in SPLs is increasing by the thousands. Hence, an automatic support is needed to deal with variability in SPL. Most of the current proposals for automatic reasoning on SPL are not devised to cope with extra–functional features. In this paper we introduce a proposal to model and reason on an SPL using constraint programming. We take into account functional and extra–functional features, improve current proposals and present a running, yet feasible implementation.

Automated analysis of feature models: challenges ahead

Current tool support for feature models is ad hoc, offering little or no support for debugging feature models or optimizing feature selections. Recent work shows how feature models can be reduced to propositional formulas or to constraint satisfaction problems, for which off-the-shelf tools can validate properties of models (e.g., confirming that a given set of features are incompatible or compatible) or to optimize the selection of features (e.g., performance) [1][2][4]. This opens up new possibilities for next-generation tools for specifying products in software product lines.

Automated Diagnosis of Product-Line Configuration Errors in Feature Models

Feature models are widely used to model software product-line (SPL) variability. SPL variants are configured by selecting feature sets that satisfy feature model constraints. Configuration of large feature models can involve multiple stages and participants, which makes it hard to avoid conflicts and errors. New techniques are therefore needed to debug invalid configurations and derive the minimal set of changes to fix flawed configurations. This paper provides three contributions to debugging feature model configurations: (1) we present a technique for transforming a flawed feature model configuration into a constraint satisfaction problem (CSP) and show how a constraint solver can derive the minimal set of feature selection changes to fix an invalid configuration, (2) we show how this diagnosis CSP can automatically resolve conflicts between configuration participant decisions, and (3) we present experiment results that evaluate our technique. These results show that our technique scales to models with over 5,000 features, which is well beyond the size used to validate other automated techniques.

Automated error analysis for the agilization of feature modeling

Software Product Lines (SPL) and agile methods share the common goal of rapidly developing high-quality software. Although they follow different approaches to achieve it, some synergies can be found between them by (i) applying agile techniques to SPL activities so SPL development becomes more agile; and (ii) tailoring agile methodologies to support the development of SPL. Both options require an intensive use of feature models, which are usually strongly affected by changes on requirements. Changing large-scale feature models as a consequence of changes on requirements is a well-known error-prone activity. Since one of the objectives of agile methods is a rapid response to changes in requirements, it is essential an automated error analysis support in order to make SPL development more agile and to produce error-free feature models. As a contribution to find the intended synergies, this article sets the basis to provide an automated support to feature model error analysis by means of a framework which is organized in three levels: a feature model level, where the problem of error treatment is described; a diagnosis level, where an abstract solution that relies on Reiters theory of diagnosis is proposed; and an implementation level, where the abstract solution is implemented by using Constraint Satisfaction Problems (CSP). To show an application of our proposal, a real case study is presented where the Feature-Driven Development (FDD) methodology is adapted to develop an SPL. Current proposals on error analysis are also studied and a comparison among them and our proposal is provided. Lastly, the support of new kinds of errors and different implementation levels for the proposed framework are proposed as the focus of our future work.

Software diversity: state of the art and perspectives

Diversity is prevalent in modern software systems to facilitate adapting the software to customer requirements or the execution environment. Diversity has an impact on all phases of the software development process. Appropriate means and organizational structures are required to deal with the additional complexity introduced by software variability. This introductory article to the special section “Software Diversity—Modeling, Analysis and Evolution” provides an overview of the current state of the art in diverse systems development and discusses challenges and potential solutions. The article covers requirements analysis, design, implementation, verification and validation, maintenance and evolution as well as organizational aspects. It also provides an overview of the articles which are part of this special section and addresses particular issues of diverse systems development.

FAMA Framework

FAMA framework (FAMA FW) is a tool for the automated analysis of variability models (VM). Its main objective is providing an extensible framework where current research on VM automated analysis might be developed and easily integrated into a final product. FAMA FW is built following the SPL paradigm supporting different variability metamodels, reasoners or solvers, analysis questions and reasoner selectors, easing the production of customized VM analysis tools. FAMA FW is written in Java and distributed under LGPL License.

BeTTy: benchmarking and testing on the automated analysis of feature models

The automated analysis of feature models is a flourishing research topic that has called the attention of both researchers and practitioners during the last two decades. During this time, the number of tools and techniques enabling the analysis of feature models has increased and also their complexity. In this scenario, the lack of specific testing mechanisms to assess the correctness and good performance of analysis tools is becoming a major obstacle hindering the development of tools and affecting their quality and reliability. In this paper, we present BeTTy, a framework for BEnchmarking and TesTing on the analYsis of feature models. Among other features, BeTTy enables the automated detection of faults in feature model analysis tools. Also, it supports the generation of motivating test data to evaluate the performance of analysis tools in both average and pessimistic cases. Part of the functionality of the framework is provided through a web-based interface facilitating the random generation of both classic and attributed feature models.

Using java CSP solvers in the automated analyses of feature models

Feature Models are used in different stages of software development and are recognized to be an important asset in model transformation techniques and software product line development. The automated analysis of feature models is being recognized as one of the key challenges for automated software development in the context of Software Product Lines. In our previous work we explained how a feature model can be transformed into a constraint satisfaction problem. However cardinalities were not considered. In this paper we present how a cardinality-based feature model can be also translated into a constraint satisfaction problem. In that connection, it is possible to use off-the-shelf tools to automatically accomplish several tasks such as calculating the number of possible feature configurations and detecting possible conflicts. In addition, we present a performance test between two off-the-shelf Java constraint solvers. To the best of our knowledge, this is the first time a performance test is presented using solvers for feature modelling proposes

Automated metamorphic testing on the analyses of feature models

Context: A feature model (FM) represents the valid combinations of features in a domain. The automated extraction of information from FMs is a complex task that involves numerous analysis operations, techniques and tools. Current testing methods in this context are manual and rely on the ability of the tester to decide whether the output of an analysis is correct. However, this is acknowledged to be time-consuming, error-prone and in most cases infeasible due to the combinatorial complexity of the analyses, this is known as the oracle problem. Objective: In this paper, we propose using metamorphic testing to automate the generation of test data for feature model analysis tools overcoming the oracle problem. An automated test data generator is presented and evaluated to show the feasibility of our approach. Method: We present a set of relations (so-called metamorphic relations) between input FMs and the set of products they represent. Based on these relations and given a FM and its known set of products, a set of neighbouring FMs together with their corresponding set of products are automatically generated and used for testing multiple analyses. Complex FMs representing millions of products can be efficiently created by applying this process iteratively. Results: Our evaluation results using mutation testing and real faults reveal that most faults can be automatically detected within a few seconds. Two defects were found in FaMa and another two in SPLOT, two real tools for the automated analysis of feature models. Also, we show how our generator outperforms a related manual suite for the automated analysis of feature models and how this suite can be used to guide the automated generation of test cases obtaining important gains in efficiency. Conclusion: Our results show that the application of metamorphic testing in the domain of automated analysis of feature models is efficient and effective in detecting most faults in a few seconds without the need for a human oracle.