Bruno Barbieri Pontes Cafeo

Quantifying usability of domain-specific languages

A usability measurement framework for DSLs was developed based on the cognitive dimensions of notations.With our proposed metrics was possible to identify DSL usability limitations.Our metrics suite revealed specific DSL features favoring maintenance tasks.Eight critical DSL usability dimensions based on the CDN framework were analyzed in this study. A domain-specific language (DSL) aims to support software development by offering abstractions to a particular domain. It is expected that DSLs improve the maintainability of artifacts otherwise produced with general-purpose languages. However, the maintainability of the DSL artifacts and, hence, their adoption in mainstream development, is largely dependent on the usability of the language itself. Unfortunately, it is often hard to identify their usability strengths and weaknesses early, as there is no guidance on how to objectively reveal them. Usability is a multi-faceted quality characteristic, which is challenging to quantify beforehand by DSL stakeholders. There is even less support on how to quantitatively evaluate the usability of DSLs used in maintenance tasks. In this context, this paper reports a study to compare the usability of textual DSLs under the perspective of software maintenance. A usability measurement framework was developed based on the cognitive dimensions of notations. The framework was evaluated both qualitatively and quantitatively using two DSLs in the context of two evolving object-oriented systems. The results suggested that the proposed metrics were useful: (1) to early identify DSL usability limitations, (2) to reveal specific DSL features favoring maintenance tasks, and (3) to successfully analyze eight critical DSL usability dimensions.

Contextual Integration Testing of Object-Oriented and Aspect-Oriented Programs: A Structural Approach for Java and AspectJ

Aspect-oriented programming (AOP) is a technology that aims to tackle software modularisation problems. Despite the benefits that may be achieved with AOP, its mechanisms represent new potential source of faults that should be handled during the test phase. In this paper a structural integration testing approach for Object-Oriented (OO) and Aspect-Oriented (AO) programs with a configurable integration depth (d) is presented. A model called CoDU (Contextual Def-Use) graph to represent the control flow and data flow between units is proposed. Based on the CoDU, a family of testing criteria is defined: all-i-nodes-Nd (control flow based), all-i-edges-Nd (control flow based), and all-i-uses-Nd (data flow based). We also present the implementation of our approach as an extension to the Java Bytecode Understanding and Testing (JaBUTi) family of testing tools along with an example of usage. The example and an exploratory study involving 8 OO and AO programs present evidence of the effectiveness and feasibility of the proposed approach.

Analysing the Impact of Feature Dependency Implementation on Product Line Stability: An Exploratory Study

The evolution of software product lines (SPLs) is particularly challenging. SPL functionalities, usually decomposed into features, naturally depend among them. As the SPL evolves, the implementation of each feature dependency might increasingly affect more code elements. Therefore, as the complexity of feature dependency implementation grows up through code evolution, it is likely to negatively impact on the SPL stability. Stability refers to the amount of changes between SPL releases and it is directly influenced by the complexity of feature dependency implementation. In this context, a number of emerging programming techniques can be used to keep the complexity of feature dependency code under control. Nevertheless, there is a lack of studies analysing the impact of different programming techniques on the stability of feature dependency code in evolving SPLs. This paper presents a first exploratory analysis on the use of programming techniques to implement and evolve feature dependencies. Our analysis was developed in the context of three evolving SPL systems, which are made up of 210 feature dependencies. Our initial findings reveal that two particular types of feature dependency implementation using different programming techniques play a role in the SPL stability.

Segregating feature interfaces to support software product line maintenance

Although software product lines are widely used in practice, their maintenance is challenging. Features as units of behaviour can be heavily scattered across the source code of a product line, hindering modular reasoning. To alleviate this problem, feature interfaces aim at enhancing modular reasoning about features. However, considering all members of a feature interface is often cumbersome, especially due to the large number of members arising in practice. To address this problem, we present an approach to group members of a feature interface based on their mutual dependencies. We argue that often only a subset of all interface members is relevant to a maintenance task. Therefore, we propose a graph representation that is able to capture the collaboration between members and apply a clustering algorithm to it to group highly-related members and segregate non-related members. On a set of ten versions of a real-world product line, we evaluate the effectiveness of our approach, by comparing the two types of feature interfaces (segregated vs. original interfaces) with co-change information from the version-control system. We found a potential reduction of 62% of the interface members to be considered during maintenance. This way, the effort to reason about features can be reduced.

Feature dependencies as change propagators

ContextA Software Product Line (SPL) is a set of software systems that share common functionalities, so-called features. When features are related, we consider this relation a feature dependency. Whenever a new feature is added, the presence of feature dependencies in the source code may increase the maintenance effort. In particular, along the maintenance of SPL implementation, added features may induce changes in other features, the so-called change propagation. Change propagation is the set of ripple changes required to other features whenever a particular feature is added or changed. ObjectiveThe relationship between feature dependency and change propagation is not well understood. Therefore, the objective of our study is to examine the relation between feature dependency and change propagation. MethodWe investigate change propagation through feature dependencies in additive changes on five evolving SPLs. We analysed a wide range of additive changes in 21 representations of those SPLs. This analysis enabled us to understand whether and how features dependencies and change propagations are related. ResultsThe results have empirically confirmed for the first time the strong relation between feature dependency and change propagation. We also identified what are the circumstances involving dependent features that are more likely to cause change propagation. Surprisingly, the results also suggested that the extent of change propagation across SPL features might be higher than the one found in previous studies of dependent modules in non-SPLs. We also found a concentration of change propagation in a few feature dependencies. ConclusionEven though the results show that there is a strong relation between feature dependencies and change propagation, such relation is not alike for all dependencies. This indicates that (i) a general feature dependency minimisation might not ameliorate the change propagation, and (ii) feature dependency properties must be analysed beforehand to drive maintenance effort to important dependencies.

Evaluation and assessment of effects on exploring mutation testing in programming courses

Mutation analysis is a testing strategy that consists of using supporting tools to seed artificial faults in the original code of a software under test, generating faulty programs (“mutants”) that are supposed to produce incorrect outputs. Novice programmers suffer of a wide range of deficits due to defective training processes. We argue that the incorporation of experiences on mutation testing in programming courses adds valuable knowledge to the learning process. In this paper we evaluate the effects of using mutation testing to improve the learning process of students in programming courses. We present results of experiments and analysis involving undergraduate students. These experiments are the continuation of a previous work in which we raise empirical evidences that the adequate incorporation of mutation testing in programming courses contributes to form an effective environment that fosters learning. To do so, we provide a mutation testing tool to promote the practice of mutation testing by novice programmers. Through practical experiences and several analysis survey we measured the effects of using the mutation testing criterion to teach programming. In addition, we collected the opinion of senior students who already knew mutation testing concepts about their opinion on the usage of mutation concepts to teach novice programmers. Our findings reveal that the effective use of mutation analysis concepts contributes to the learning process, making students see the code as a product under development that is the result of a careful manual coding process which they need for measuring and predicting the effect of each command. The main contributions discussed in this paper are: (1) presenting results of an empirical analysis involving undergraduate students, thus giving us preliminary evidence on the effects of the novel practice; (2) exposing possible practices to explore mutation testing in programming classes, highlighting the limitations and strengths of such strategy; and (3) a mutation testing tool for educational purposes.

Testing of aspect-oriented programs: difficulties and lessons learned based on theoretical and practical experience

BackgroundSince the first discussions of new challenges posed by aspect-oriented programming (AOP) to software testing, the real difficulties of testing aspect-oriented (AO) programs have not been properly analysed. Firstly, despite the customisation of traditional testing techniques to the AOP context, the literature lacks discussions on how hard it is to apply them to (even ordinary) AO programs based on practical experience. Secondly, and equally important, due to the cautious AOP adoption focused on concern refactoring, test reuse is another relevant issue that has been overlooked so far. This paper deals with these two issues. It discusses the difficulties of testing AO programs from three perspectives: (i) structural-based testing, (ii) fault-based testing and (iii) test set reuse across paradigms.MethodsPerspectives (i) and (ii) are addressed by means of a retrospective of research done by the authors’ group. We analyse the impact of using AOP mechanisms on the testability of programs in terms of the underlying test models, the derived test requirements and the coverage of such requirements. The discussion is based on our experience on developing and applying testing approaches and tools to AspectJ programs at both unit and integration levels. Perspective (iii), on the other hand, consists of recent exploratory studies that analyse the effort to adapt test sets for refactored systems and the quality of such test sets in terms of structural coverage.ResultsBuilding test models for AO programs imposes higher complexity when compared to the OO paradigm. Besides this, adapting test suites for OO programs to AO equivalent programs tends to require less effort than doing the other way around, and resulting suites achieve similar quality levels for small-sized aplications.ConclusionsThe conclusion is that building test models for AO programs, as well as deriving and covering paradigm-specific test requirements, is not straightforward as it has been for procedural and object-oriented (OO) programs at some extent. Once you have test suites in conformance with programs implemented in both paradigms, the quality of such suited in termos of code coverage may vary depending on the size and characteristics of the applications under testing.

Towards indicators of instabilities in software product lines: An empirical evaluation of metrics

A Software Product Line (SPL) is a set of software systems (products) that share common functionalities, so-called features. The success of a SPL design is largely dependent on its stability; otherwise, a single implementation change will cause ripple effects in several products. Therefore, there is a growing concern in identifying means to either indicate or predict design instabilities in the SPL source code. However, existing studies up to now rely on conventional metrics as indicators of SPL instability. These conventional metrics, typically used in standalone systems, are not able to capture the properties of SPL features in the source code, which in turn might neglect frequent causes of SPL instabilities. On the other hand, there is a small set of emerging software metrics that take into account specific properties of SPL features. The problem is that there is a lack of empirical validation of the effectiveness of metrics in indicating quality attributes in the context of SPLs. This paper presents an empirical investigation through two set of metrics regarding their power of indicating instabilities in evolving SPLs. A set of conventional metrics was confronted with a set of metrics we instantiated to capture important properties of SPLs. The software evolution history of two SPLs were analysed in our studies. These SPLs are implemented using two different programming techniques and all together they encompass 30 different versions under analysis. Our analysis confirmed that conventional metrics are not good indicators of instabilities in the context of evolving SPLs. The set of employed feature dependency metrics presented a high correlation with instabilities proving its value as indicator of SPL instabilities.

Inferring test results for dynamic software product lines

Due to the very large number of configurations that can typically be derived from a Dynamic Software Product Line (DSPL), efficient and effective testing of such systems have become a major challenge for software developers. In particular, when a configuration needs to be deployed quickly due to rapid contextual changes (e.g., in an unfolding crisis), time constraints hinder the proper testing of such a configuration. In this paper, we propose to reduce the testing required of such DSPLs to a relevant subset of configurations. Whenever a need to adapt to an untested configuration is encountered, our approach determines the most similar tested configuration and reuses its test results to either obtain a coverage measure or infer a confidence degree for the new, untested configuration. We focus on providing these techniques for inference of structural testing results for DSPLs, which is supported by an early prototype implementation.

Comprehensibility of Heterogeneous Configuration Knowledge: A User Study

An essential part of configurable software systems is the configuration knowledge (CK), which is used by developers to derive customized products. Proper comprehension of the CK is key to achieve correct product customization. However, each configurable system is often built with heterogeneous software technologies, such as diverse types of frameworks. These technologies impose the use of additional concepts on the definition of the CK, thereby making it harder to comprehend. This paper presents a domain specific modeling technique aimed at improving the comprehensibility of the CK in framework-based configurable systems. We also present the results of a user study that compares domain-specific modeling with two other state-of-the-art CK techniques, namely annotation-based and general-purpose modeling. We analyzed the impact of these techniques on the CK comprehensibility across three configurable systems. The results strongly suggest that: (i) domain-specific models improve the CK comprehensibility; (ii) the use of general purpose models imposes significant obstacles to the CK comprehension; and (iii) participants who worked with domain-specific models rated the comprehension tasks as easier than those that worked without these models.