Yoav Katz

X-Gen: a random test-case generator for systems and SoCs

We present X-Gen, a model-based test-case generator designed for systems and systems on a chip (SoC). X-Gen provides a framework and a set of building blocks for system-level test-case generation. At the core of this framework lies a system model, which consists of component types, their configuration, and the interactions between them. Building blocks include commonly used concepts such as memories, registers, and address translation mechanisms. Once a system is modeled, X-Gen provides a rich language for describing test cases. Through this language, users can specify requests that cover the full spectrum between highly directed tests to completely random ones. X-Gen is currently in preliminary use at IBM for the verification of two different designs - a high-end multi-processor server and a state-of-the-art SoC.

Constraint-based random stimuli generation for hardware verification

We report on random stimuli generation for hardware verification at IBM as a major applica-tion of various artificial intelligence technologies, including knowledge representation, expert systems, and constraint satisfaction. For more than a decade we have developed several related tools, with huge payoffs. Research and development around this application are still thriving, as we continue to cope with the ever-increasing complexity of modern hardware systems and demanding business environments.

Learning microarchitectural behaviors to improve stimuli generation quality

Microarchitectural information regarding various aspects of instruction execution can help processor-level stimuli generators more easily reach verification goals. While many such aspects are based on common microarchitectural concepts, their specific manifestations are highly design-specific. We propose using an automatic method for acquiring such microarchitectural knowledge and integrating it into the stimuli generator. We start by extracting microarchitectural data from simulation traces. This data is fed to a decision tree learning algorithm that produces rules for microarchi-tectural behavior of instructions; these rules are then integrated into the testing knowledge of the stimuli generator. This testing knowledge can provide users with the ability to better control the microarchitectural behavior of generated instructions, leading to higher quality test cases. Experimental results on the POWER7 processor showed that our proposed method can improve the microarchitectural cover-age of the design

DeepTrans - a model-based approach to functional verification of address translation mechanisms

We present a new test case generation technology, specifically targeted at verifying systems that include address translation mechanisms. The ever-growing demand for performance makes these mechanisms more complex, thereby increasing the risk of bugs and increasing the need for such technology. DeepTrans is a package that provides model-based test generation capabilities to verify translation mechanisms based on a modeling language. The modeling language includes constructs for describing the address translation process, commonly used translation resources, and architecture rules related to translation. DeepTrans is currently used by two different IBM test generators.

Generating instruction streams using abstract CSP

One of the challenges that processor level stimuli generators are facing is the need to generate stimuli that exercise microarchitectural mechanisms deep inside the verified processor. These scenarios require specific relations between the instructions participating in them. We present a new approach for processor-level scenario generation. The approach is based on creating an abstract constraint satisfaction problem, which captures the essence of the requested scenario. The generation of stimuli is done by interleaving between progress in the solution of the abstract CSP and generation of instructions. Compared with existing solutions of scenario generation, this approach yields improved coverage and reduced generation fail rate.

DeepTrans - Extending the Model-based Approach to Functional Verification of Address Translation Mechanisms

This paper presents a new test case generation technology, specifically targeted at verifying systems that include address translation mechanisms. The ever-growing demand for performance makes these mechanisms more complex, thereby increasing the risk of bugs and increasing the need for such technology. DeepTrans is a package that extends existing test generators with address translation testing capabilities. It uses a declarative modeling language that includes constructs for describing the address translation process, commonly used translation resources, and architecture rules related to translation. The address translation model is converted to a constraint satisfaction problem that is solved simultaneously with the problem formulated by the generator. DeepTrans is currently used by two different IBM test generators

Path-Based system level stimuli generation

Over the last few years, there has been increasing emphasis on integrating ready-made components (IP, cores) into complex System on a Chip (SoC) designs. The verification of such designs poses new challenges. At the heart of these challenges lies the requirement to verify the integration of several previously designed components in a relatively short time. Simulation-based methods are the main verification vehicle used for system-level functional verification of SoC designs; therefore, stimuli generation plays an important role in this field. Our work offers a solution for efficiently dealing with the verification of systems with multiple configurations and derivative systems, a common challenge in the context of system verification. We present a generation scheme in which the system behavior is defined using a combination of transaction-based modeling, local component behavior, and the topology of the system. We show how this approach allows the implementation of the verification plan using high level constructs and promotes the reuse of verification IP between systems. The ideas described below were implemented as part of X-Gen, a system-level test-case generator developed and used in IBM.

A novel approach for implementing microarchitectural verification plans in processor designs

The ever-growing microarchitecture complexity of processors creates a widening gap between the verification plan and the test generation technologies used in its implementation. This gap impacts the cost and quality of the verification process. To overcome this, we introduce a novel test generation platform for processor verification. This approach is based on a scenario description language that is close to the microarchitecture verification plan, and uses new test generation algorithms and a microarchitectural model to support this higher level of abstraction. Initial results on a high end industrial design show our approach reduces the effort of implementing a microarchitectural verification plan and improves the quality of verification.

Enhancing Scenario Quality Using Quasi-Events

A major challenge for processor-level stimuli generators is the need to generate stimuli that exercise deep micro-architectural mechanisms. Advanced generators address this challenge by applying expert testing knowledge that bias the stimuli toward interesting verification events. In this paper, we present a new approach whereby scenarios are not just enhanced, but are actually modified by testing knowledge. By allowing such mutations, scenarios are diverted toward quasi-events that are semantically related, though not identical, to the original intent of the scenario. We describe the importance of quasi-events and the usefulness of automated scenario mutations for improving the verification of speculative execution.