Shoham Ben-David

Attention-Based Coverage Metrics

Over the last decade, extensive research has been conducted on coverage metrics for model checking. The most common coverage metrics are based on mutations, where one examines the effect of small modifications of the system on the satisfaction of the specification. While it is commonly accepted that mutation-based coverage provides adequate means for assessing the exhaustiveness of the model-checking procedure, the incorporation of coverage checks in industrial model checking tools is still very partial. One reason for this is the typically overwhelming number of non-covered mutations, which requires the user to somehow filter those that are most likely to point to real errors or overlooked behaviors.

The Temporal Logic Sugar

Since the introduction of temporal logic for the specification of computer programs [5], usability has been an issue, because a difficult-to-use formalism is a barrier to the wide adoption of formal methods. Our solution is Sugar, the temporal logic used by the RuleBase formal verification tool [2]. Sugar adds the power of regular expressions to CTL [4], as well as an extensive set of operators which provide syntactic sugar. That is, while these operators do not add expressive power, they allow properties to be expressed more succinctly than in the basic language. Experience shows that Sugar allows hardware engineers to easily and intuitively specify their designs. The full language is used for model checking, and a significant portion can be model checked on-the-fly [3]. The automatic generation of simulation checkers from the same portion of Sugar is described in [1]. While previous papers have described various features of the language, this paper presents the first complete description of Sugar.

RuleBase: an industry-oriented formal verification tool

RuleBase is a formal verification tool, developed by the IBM Haifa Research Laboratory. It is the result of three years of experience in practical formal verification of hardware which, we believe, has been a key factor in bringing the tool to its current level of maturity. We present the tool, including several unique features, and summarize our usage experience.

Efficient Detection of Vacuity in ACTL Formulas

Prepositional logic formulas containing implications can suffer from antecedent failure, in which the formula is true trivially because the pre-condition of the implication is not satisfiable. In other words, the post-condition of the implication does not affect the truth value of the formula. We call this a vacuous pass, and extend the definition of vacuity to cover other kinds of trivial passes in temporal logic. We define w-ACTL, a subset of CTL and show by construction that for every w-ACTL formula ϕ there is a formula w(ϕ), such that: both ϕ and w(ϕ) are true in some model M iff ϕ passes vacuously. A useful side-effect of w(ϕ) is that if false, any counter-example is also a non-trivial witness of the original formula ϕ.

Efficient Detection of Vacuity in Temporal Model Checking

The ability to generate a counter-example is an important feature of model checking tools, because a counter-example provides information to the user in the case that the formula being checked is found to be non-valid. In this paper, we turn our attention to providing similar feedback to the user in the case that the formula is found to be valid, because valid formulas can hide real problems in the model. For instance, propositional logic formulas containing implications can suffer from antecedent failure, in which the formula is trivially valid because the pre-condition of the implication is not satisfiable. We call this vacuity, and extend the definition to cover other kinds of trivial validity. For non-vacuously valid formulas, we define an interesting witness as a non-trivial example of the validity of the formula. We formalize the notions of vacuity and interesting witness, and show how to detect vacuity and generate interesting witnesses in temporal model checking. Finally, we provide a practical solution for a useful subset of ACTL formulas.

On-the-Fly Model Checking of RCTL Formulas

The specification language RCTL, an extension of CTL, is defined by adding the power of regular expressions to CTL. In addition to being a more expressive and natural hardware specification language than CTL, a large family of RCTL formulas can be verified on-the-fly (during symbolic reachability analysis). On-the-fly model checking, as a powerful verification paradigm, is especially efficient when the specification is false and extremely efficient when the computation needed to get to a failing state is short. It is suitable for the inherently gradual design process since it detects a multitude of bugs at the early verification stages, and paves the way towards finding the more complex errors as the design matures. It is shown that for every erroneous finite computation, there is an RCTL formula that detects it and can be verified on-the-fly. On-the-fly verification of RCTL formulas has moved model checking in IBM into a different class of designs inaccessible by prior techniques.

Methodology and System for Practical Formal Verification of Reactive Hardware

Making formal verification a practicality in industrial environments is still difficult. The capacity of most verification tools is too small, their integration in a design process is difficult and the methodology that should guide their usage is unclear.

Explaining Counterexamples Using Causality

When a model does not satisfy a given specification, a counterexample is produced by the model checker to demonstrate the failure. A user must then examine the counterexample trace, in order to visually identify the failure that it demonstrates. If the trace is long, or the specification is complex, finding the failure in the trace becomes a non-trivial task. In this paper, we address the problem of analyzing a counterexample trace and highlighting the failure that it demonstrates. Using the notion of causality , introduced by Halpern and Pearl, we formally define a set of causes for the failure of the specification on the given counterexample trace. These causes are marked as red dots and presented to the user as a visual explanation of the failure. We study the complexity of computing the exact set of causes, and provide a polynomial-time algorithm that approximates it. This algorithm is implemented as a feature in the IBM formal verification platform RuleBase PE, where these visual explanations are an integral part of every counterexample trace. Our approach is independent of the tool that produced the counterexample, and can be applied as a light-weight external layer to any model checking tool, or used to explain simulation traces.

RuleBase: Model Checking at IBM

RuleBase is a symbolic model checking tool, developed by the IBM Haifa Research Laboratory. It is the result of four years of experience in practical formal verification of hardware which, we believe, has been a key factor in bringing the tool to its current level of maturity. Our experience shows that after a short training period, designers can operate the tool independently and achieve impressive results. We present the tool and summarize our development and usage experience, focusing on some work done during 1996.

Model Checking at IBM

Over the past nine years, the Formal Methods Group at the IBM Haifa Research Laboratory has made steady progress in developing tools and techniques that make the power of model checking accessible to the community of hardware designers and verification engineers, to the point where it has become an integral part of the design cycle of many teams. We discuss our approach to the problem of integrating formal methods into an industrial design cycle, and point out those techniques which we have found to be especially effective in an industrial setting.