Bihuan Chen

Self-tuning of software systems through dynamic quality tradeoff and value-based feedback control loop

Quality requirements of a software system cannot be optimally met, especially when it is running in an uncertain and changing environment. In principle, a controller at runtime can monitor the change impact on quality requirements of the system, update the expectations and priorities from the environment, and take reasonable actions to improve the overall satisfaction. In practice, however, existing controllers are mostly designed for tuning low-level performance indicators instead of high-level requirements. By maintaining a live goal model to represent runtime requirements and linking the overall satisfaction of quality requirements to an indicator of earned business value, we propose a control-theoretic self-tuning method that can dynamically tune the preferences of different quality requirements, and can autonomously make tradeoff decisions through our Preference-Based Goal Reasoning procedure. The reasoning procedure results in an optimal configuration of the variation points by selecting the right alternative of OR-decomposed goals and such a configuration is mapped onto corresponding system architecture reconfigurations. The effectiveness of our self-tuning method is evaluated by earned business value, comparing our results with those obtained using static and ad hoc methods.

Self-adaptation through incremental generative model transformations at runtime

A self-adaptive system uses runtime models to adapt its architecture to the changing requirements and contexts. However, there is no one-to-one mapping between the requirements in the problem space and the architectural elements in the solution space. Instead, one refined requirement may crosscut multiple architectural elements, and its realization involves complex behavioral or structural interactions manifested as architectural design decisions. In this paper we propose to combine two kinds of self-adaptations: requirements-driven self-adaptation, which captures requirements as goal models to reason about the best plan within the problem space, and architecture-based self-adaptation, which captures architectural design decisions as decision trees to search for the best design for the desired requirements within the contextualized solution space. Following these adaptations, component-based architecture models are reconfigured using incremental and generative model transformations. Compared with requirements-driven or architecture-based approaches, the case study using an online shopping benchmark shows promise that our approach can further improve the effectiveness of adaptation (e.g. system throughput in this case study) and offer more adaptation flexibility.

Self-Tuning of Software Systems Through Goal-based Feedback Loop Control

Quality requirements of a software system cannot be optimally met, especially when it is running in an uncertain and changing environment. In principle, a controller at runtime can monitor the change impact on quality requirements of the system, update the expectations and priorities from the environment, and take reasonable actions to improve the overall satisfaction. In practice, however, existing controllers are mostly designed for tuning low-level performance indicators rather than high-level requirements. By linking the overall satisfaction to a business value indicator as feedback, we propose a control theoretic self-tuning method that can dynamically adjust the tradeoff decisions among different quality requirements. A preference-based reasoning algorithm is involved to configure hard goals accordingly to guide the following architecture reconfiguration.

Steelix: program-state based binary fuzzing

Coverage-based fuzzing is one of the most effective techniques to find vulnerabilities, bugs or crashes. However, existing techniques suffer from the difficulty in exercising the paths that are protected by magic bytes comparisons (e.g., string equality comparisons). Several approaches have been proposed to use heavy-weight program analysis to break through magic bytes comparisons, and hence are less scalable. In this paper, we propose a program-state based binary fuzzing approach, named Steelix, which improves the penetration power of a fuzzer at the cost of an acceptable slow down of the execution speed. In particular, we use light-weight static analysis and binary instrumentation to provide not only coverage information but also comparison progress information to a fuzzer. Such program state information informs a fuzzer about where the magic bytes are located in the test input and how to perform mutations to match the magic bytes efficiently. We have implemented Steelix and evaluated it on three datasets: LAVA-M dataset, DARPA CGC sample binaries and five real-life programs. The results show that Steelix has better code coverage and bug detection capability than the state-of-the-art fuzzers. Moreover, we found one CVE and nine new bugs.

Are your sites down? Requirements-driven self-tuning for the survivability of Web systems

Running in a highly uncertain and greatly complex environment, Web systems cannot always provide full set of services with optimal quality, especially when work loads are high or subsystem failures are frequent. Hence, it is significant to continuously maintain a high satisfaction level of survivability, hereafter survivability assurance, while relaxing or sacrificing certain quality or functional requirements that are not crucial to the survival of the entire system. After giving a value-based interpretation to survivability assurance to facilitate a quantitative analysis, we propose a requirements-driven self-tuning method for the survivability assurance of Web systems. Maintaining an enriched and live goal model, our method adapts to runtime tradeoff decisions made by our PID (proportional-integral-derivative) controller and goal-oriented reasoner for both quality and functional requirements. The goal-based configuration plans produced by the reasoner is carried out on the live goal model, and then mapped into system architectural configurations. Experiments on an online shopping system are conducted to validate the effectiveness of the proposed method.

Skyfire: Data-Driven Seed Generation for Fuzzing

Programs that take highly-structured files as inputs normally process inputs in stages: syntax parsing, semantic checking, and application execution. Deep bugs are often hidden in the application execution stage, and it is non-trivial to automatically generate test inputs to trigger them. Mutation-based fuzzing generates test inputs by modifying well-formed seed inputs randomly or heuristically. Most inputs are rejected at the early syntax parsing stage. Differently, generation-based fuzzing generates inputs from a specification (e.g., grammar). They can quickly carry the fuzzing beyond the syntax parsing stage. However, most inputs fail to pass the semantic checking (e.g., violating semantic rules), which restricts their capability of discovering deep bugs. In this paper, we propose a novel data-driven seed generation approach, named Skyfire, which leverages the knowledge in the vast amount of existing samples to generate well-distributed seed inputs for fuzzing programs that process highly-structured inputs. Skyfire takes as inputs a corpus and a grammar, and consists of two steps. The first step of Skyfire learns a probabilistic context-sensitive grammar (PCSG) to specify both syntax features and semantic rules, and then the second step leverages the learned PCSG to generate seed inputs. We fed the collected samples and the inputs generated by Skyfire as seeds of AFL to fuzz several open-source XSLT and XML engines (i.e., Sablotron, libxslt, and libxml2). The results have demonstrated that Skyfire can generate well-distributed inputs and thus significantly improve the code coverage (i.e., 20% for line coverage and 15% for function coverage on average) and the bug-finding capability of fuzzers. We also used the inputs generated by Skyfire to fuzz the closed-source JavaScript and rendering engine of Internet Explorer 11. Altogether, we discovered 19 new memory corruption bugs (among which there are 16 new vulnerabilities and received 33.5k USD bug bounty rewards) and 32 denial-of-service bugs.

Rationalism with a dose of empiricism: Case-based reasoning for requirements-driven self-adaptation

Requirements-driven approaches provide an effective mechanism for self-adaptive systems by reasoning over their runtime requirements models to make adaptation decisions. However, such approaches usually assume that the relations among alternative behaviours, environmental parameters and requirements are clearly understood, which is often simply not true. Moreover, they do not consider the influence of the current behaviour of an executing system on adaptation decisions. In this paper, we propose an improved requirements-driven self-adaptation approach that combines goal reasoning and case-based reasoning. In the approach, past experiences of successful adaptations are retained as adaptation cases, which are described by not only requirements violations and contexts, but also currently deployed behaviours. The approach does not depend on a set of original adaptation cases, but employs goal reasoning to provide adaptation solutions when no similar cases are available. And case-based reasoning is used to provide more precise adaptation decisions that better reflect the complex relations among requirements violations, contexts, and current behaviours by utilizing past experiences. Our experimental study with an online shopping benchmark shows that our approach outperforms both requirements-driven approach and case-based reasoning approach in terms of adaptation effectiveness and overall quality of the system.

Requirements-Driven Self-Optimization of Composite Services Using Feedback Control

In an uncertain and changing environment, a composite service needs to continuously optimize its business process and service selection through runtime adaptation. To achieve the overall satisfaction of stakeholder requirements, quality tradeoffs are needed to adapt the composite service in response to the changing environments. Existing approaches on service selection and composition, however, are mostly based on quality preferences and business processes decisions made statically at the design time. In this paper, we propose a requirements-driven self-optimization approach for composite services. It measures the quality of services (QoS), estimates the earned business value, and tunes the preference ranks through a feedback loop. The detection of unexpected earned business value triggers the proposed self-optimization process systematically. At the process level, a preference-based reasoner configures a requirements goal model according to the tuned preference ranks of QoS requirements, reconfiguring the business process according to its mappings from the goal configurations. At the service level, selection decisions are optimized by utilizing the tuned weights of QoS criteria. We used an experimental study to evaluate the proposed approach. Results indicate that the new approach outperforms both fixed-weighted and floating-weighted service selection approaches with respect to earned business value and adaptation flexibility.

Proteus: computing disjunctive loop summary via path dependency analysis

Loops are challenging structures for program analysis, especially when loops contain multiple paths with complex interleaving executions among these paths. In this paper, we first propose a classification of multi-path loops to understand the complexity of the loop execution, which is based on the variable updates on the loop conditions and the execution order of the loop paths. Secondly, we propose a loop analysis framework, named Proteus, which takes a loop program and a set of variables of interest as inputs and summarizes path-sensitive loop effects on the variables. The key contribution is to use a path dependency automaton (PDA) to capture the execution dependency between the paths. A DFS-based algorithm is proposed to traverse the PDA to summarize the effect for all feasible executions in the loop. The experimental results show that Proteus is effective in three applications: Proteus can 1) compute a more precise bound than the existing loop bound analysis techniques; 2) significantly outperform state-of-the-art tools for loop verification; and 3) generate test cases for deep loops within one second, while KLEE and Pex either need much more time or fail.

Uncertainty handling in goal-driven self-optimization - Limiting the negative effect on adaptation

Graphical abstractDisplay Omitted HighlightsWe propose techniques to handle contribution uncertainty and effect uncertainty in goal-driven self-optimization.We integrate these uncertainty handling techniques with preference uncertainty handling to a goal-driven self-optimization framework.We demonstrate the effectiveness of our approach with an evaluation on an online shopping system. Goal-driven self-optimization through feedback loops has shown effectiveness in reducing oscillating utilities due to a large number of uncertain factors in the runtime environments. However, such self-optimization is less satisfactory when there contains uncertainty in the predefined requirements goal models, such as imprecise contributions and unknown quality preferences, or during the switches of goal solutions, such as lack of understanding about the time for the adaptation actions to take effect. In this paper, we propose to handle such uncertainty in goal-driven self-optimization without interrupting the services. Taking the monitored quality values as the feedback, and the estimated earned value as the global indicator of self-optimization, our approach dynamically updates the quantitative contributions from alternative functionalities to quality requirements, tunes the preferences of relevant quality requirements, and determines a proper timing delay for the last adaptation action to take effect. After applying these runtime measures to limit the negative effect of the uncertainty in goal models and their suggested switches, an experimental study on a real-life online shopping system shows the improvements over goal-driven self-optimization approaches without uncertainty handling.