Anca I. Vermesan

Validation and Verification of Knowledge Based Systems

Knowledge Based Systems; Validation & Verification; Software Reliability; Methods of Processes; Requirements. The Quality Assessment Process is in charge of assuring several performances of the software products and processes. One of them is the Software Reliability, perhaps the closest concept to the more general of Quality. These Quality processes should be integrated with the product development and depend mainly on two issues. One is the development methodology and the other is the type of product. The Knowledge Based Systems (KBS) are products whose software production presents special difficulties for the Validation and Verification (V &) process application. Indeed, the knowledge characteristics they model, often incomplete, imprecise, inconsistent and uncertain, are solved by the application of heuristics whose validation is difficult to carry out without the systematic application of test cases. This makes that often the development of KBS goes through a life cycle based on prototypes to build a consistent core that can be modelled conceptually and formally. This article aims to describe the set of V & V process activities that can be applied to a KBS core, that is, the essential components of a V & V Plan. Indeed although this Plan is often not defined, it is essential to assure the reliability of the system requirements, result of the prototyping phase. This paper was written with the support and financing of the Spanish Council for Research and Technology (CICYT) [Project TIC96-0883-CE]

Software certification for industry-verification and validation issues in expert systems

Research in verification and validation (V&V) of knowledge based systems (KBS) has emerged as a distinct field in the last decade and is intended to address issues associated with KBS quality aspects and to credit such applications to the same degree as conventional applications. Part of the knowledge and techniques developed are now ready to be transferred to industry for the implementation of software certification. Certification is probably the most advanced use of the V&V methods, techniques and knowledge that have been developed so far. The paper approaches the challenging issues of certifying the knowledge base (KB) component of a KBS. KB certification uses a number of different certification methods such as static analysis, testing, inspection, and modeling to assess the quality of the KB component. Although these methods are currently used in KBS development, it is not yet clear how they would be applied by an independent certification organization to arrive at a pass/fail decision. The article discusses the components of KB certification and identifies application areas directly connected with industrial needs.

Validation and verification of knowledge-based systems: report on EUROVAV99

Knowledge-Based (KB) technology is being applied to complex problem solving and safety and business critical tasks in many application domains. Concerns have naturally arisen as to the dependability of Knowledge-Based Systems (KBS). As with any software, attention to quality and safety must be paid throughout development of a KBS, and rigorous Verification and Validation (V&V) techniques must be employed. Research in V&V of KBSs has emerged as a distinct field only in the last decade, and is intended to address issues associated with quality and safety aspects of KBSs, and to provide such applications with the same degree of dependability as conventional applications. In recent years, V&V of KBSs has been the topic of annual workshops associated with the main AI conferences, such as AAAI, IJCAI and ECAI.

Techniques for the verification and validation of knowledge-based systems: A survey based on the symbol/knowledge level distinction

Verification and validation are terms that have been used for several years in software engineering, and there are now many verification and validation techniques, plus considerable experience and expertise in using them. However, applying these techniques to knowledge‐based system is not straightforward. The essential differences between conventional systems and knowledge‐based systems suggest that these techniques must be expanded and adapted, but new techniques are also needed. This article has two major goals: first, it makes some comparisons between verification and validation as found in traditional software engineering and knowledge‐based systems, pointing out what is special about the latter as compared with the former; second, it provides a framework for a discussion of the various European work on verification and validation of knowledge‐based systems. The perspective put forward in this article allows for a vast amound of work to be surveyed and analysed beyond the implementation level, by differentiating the symbol level and the knowledge level within a knowledge‐based system.

Applicability of Conventional Software Verification and Validation to Knowledge-Based Components

Verification, validation and testing techniques developed for use with conventional development practices, are not always applicable when developing knowledge-based software. This paper presents an experimental framework to determine whether a technique is applicable or not, based on concepts from mutation testing. The framework itself comprises of a number of steps guiding the researcher/practitioner in the assessment process. Mutation testing is used to simulate faults in an example programme to determine the technique’s ability to detect them. The framework has been applied to two techniques: control-flow analysis and cause-effect graphing. The conclusion is that the framework gives a good basis for a qualitative assessment of the applicability and efficiency of applying specific traditional VV&T techniques to knowledge-based components.

Verification and Validation in Support for Software Certification Methods

Companies are introducing more and more intricate software into systems. In complex systems, regulatory or other considerations (such as potential financial losses) may require the software components to undergo a certain level of certification corresponding to its application and criticality. This paper presents an approach to certifying software contained within systems where its functionality is considered critical within the environment in which it is placed. Those software components implemented using the knowledge-base (KB) technology are highlighted and the role of verification and validation (V&V) in certification of KB components is emphasized. Within a certification model, two particular elements are presented in detail. Firstly, a categorization scheme based on V&V classes is described which has the potential for ensuring reproducibility of the results from software certification by different bodies. Secondly, a certification profile for a software component is described which aims to ensure repeatability of the results of software certification carried out by the same body. The approach presented here is based on the findings of the SafeKBS1 European project.