Derek Partridge

Software diversity: practical statistics for its measurement and exploitation

Abstract The topic of this paper is the exploitation of diversity to enhance computer system reliability. It is well established that a diverse system composed of multiple alternative versions is more reliable than any single version alone, and this knowledge has occasionally been exploited in safety-critical applications. However, it is not clear what this property is, nor how the available diversity in a collection of versions is best exploited. We develop, define, illustrate and assess diversity measures, voting strategies for diversity exploitation, and interactions between the two. We take the view that a proper understanding of such issues is required if multiversion software engineering is to be elevated from the current “try it and see” procedure to a systematic technology. In addition, we introduce inductive programming techniques, particularly neural computing, as a cost-effective route to the practical use of multiversion systems outside the demanding requirements of safety-critical systems, i.e. in general software engineering.

Journeys in non-classical computation I: A grand challenge for computing research

1. The challengeA gateway event [35] is a change to a system that leads to the possibility of huge increases inkinds and levels of complexity. It opens up a whole new kind of phase space to the system’sdynamics.Gatewayeventsduringevolutionoflifeonearthincludetheappearanceofeukaryotes(organisms with a cell nucleus), an oxygen atmosphere, multi-cellular organisms and grass.Gatewayeventsduringthedevelopmentofmathematicsincludeeachinventionofanewclassofnumbers (negative, irrational, imaginary, ...), and dropping Euclid’s parallel postulate.A gateway event produces a profound and fundamental change to the system: Oncethrough the gateway, life is never the same again. We are currently poised on the threshold ofa significant gateway event in computation: That of breaking free from many of our current“classical computational” assumptions. Our Grand Challenge for computer science isto journey through the gateway event obtained by breaking our current classicalcomputational assumptions, and thereby develop a mature science of Non-ClassicalComputation2. Journeys versus goals

Assessing the impact of input features in a feedforward neural network

For a variety of reasons, the relative impacts of neural-net inputs on the output of a network’s computation is valuable information to obtain. In particular, it is desirable to identify the significant features, or inputs, of a data-defined problem before the data is sufficiently preprocessed to enable high performance neural-net training. We have defined and tested a technique for assessing such input impacts, which will be compared with a method described in a paper published earlier in this journal. The new approach, known as the ‘clamping’ technique, offers efficient impact assessment of the input features of the problem. Results of the clamping technique prove to be robust under a variety of different network configurations. Differences in architecture, training parameter values and subsets of the data all deliver much the same impact rankings, which supports the notion that the technique ranks an inherent property of the available data rather than a property of any particular feedforward neural network. The success, stability and efficiency of the clamping technique are shown to hold for a number of different real-world problems. In addition, we subject the previously published technique, which we will call the ‘weight product’ technique, to the same tests in order to provide directly comparable information.

Network generalization differences quantified

Abstract It has long been observed, and frequently noted, by connectionists that small changes in initial conditions, prior to training, can result in networks that generalize very differently. We have performed a systematic study of this phenomenon, using a number of different statistical measures of generalization differences. From these we derive a formal definition of Generalization Diversity . We quantify the relative impacts on generalization of the major parameters used in network initialization as well as extend the formal framework to also encompass the differences in generalization difference from one parameter to another. We reveal, for example, the relative effects of random initialization of the link weights and variation of the number of hidden units, and how similar these two resultant effects are. Finally, examples are presented of how the proposed generalization diversity measure may be exploited in order to improve the performance of neural-net systems. We show how several of these measures can be used to engineer reliability improvements in neural-net systems.

Diversity between Neural Networks and Decision Trees for Building Multiple Classifier Systems

A multiple classifier system can only improve the performance when the members in the system are diverse from each other. Combining some methodologically different techniques is considered a constructive way to expand the diversity. This paper investigates the diversity between the two different data mining techniques, neural networks and automatically induced decision trees. Input decimation through salient feature selection is also explored in the paper in the hope of acquiring further diversity. Among various diversities defined, the coincident failure diversity (CFD) appears to be an effective measure of useful diversity among classifiers in a multiple classifier system when the majority voting decision strategy is applied. A real-world medical classification problem is presented as an application of the techniques. The constructed multiple classifier systems are evaluated with a number of statistical measures in terms of reliability and generalisation. The results indicate that combined MCSs of the nets and trees trained with the selected features have higher diversity and produce better classification results.

Creativity: a survey of AI approaches

AbstractIn this paper we critically survey the AI programs that have been developed to exhibit some aspect of creative behaviour. We describe five necessary characteristics of models of creativity, and we apply these characteristics to help assess the programs surveyed. These characteristic features also provide a basis for a new theory of creative behavior: an emergent memory model. The survey is concluded with an assessment of an implementation of this latest theory. Bill sings to Sarah. Sarah sings to Bill. Perhaps they will do other dangerous things together. They may eat lamb or stroke each other. They may chant of their difficulties and their happiness. They have love but they also have typewriters. That is interesting.

The scope and limitations of first generation expert systems

Abstract It is clear that expert systems technology is one of AIs greatest successes so far. Currently we see an ever increasing application of expert systems, with no obvious limits to their applicability. Yet there are also a number of well-recognized problems associated with this new technology. I shall argue that these problems are not the puzzles of normal science that will yield to advances within the current technology; on the contrary, they are symptoms of severe inherent limitations of this first generation technology. By reference to these problems I shall outline some important aspects of the scope and limitations of current expert systems technology. The recognition of these limitations is a prerequisite of overcoming them as well as of developing an awarenness of the scope of applicability of this new technology.

Journeys in non-classical computation II: initial journeys and waypoints

†University of York, Heslington, York YO1 5DD, UK‡University of the West of England, Frenchay Campus, Coldharbourlane, Bristol BS16 1QY, UK{University of Portsmouth, Buckingham Building, Lion Terrace, Portsmouth PO1 3WE, UK§University of Kent, Canterbury, Kent CT2 7NZ, UKkUniversity of Cambridge, Madingley Road, Cambridge CB3 OHE, UK#University of Exeter, North Cote House, The Queen’s Drive, Exeter E4 4QJ, UK

A Comparative Study of Feature-Salience Ranking Techniques

We assess the relative merits of a number of techniques designed to determine the relative salience of the elements of a feature set with respect to their ability to predict a category outcome-for example, which features of a character contribute most to accurate character recognition. A number of different neural-net-based techniques have been proposed (by us and others) in addition to a standard statistical technique, and we add a technique based on inductively generated decision trees. The salience of the features that compose a proposed set is an important problem to solve efficiently and effectively, not only for neural computing technology but also in order to provide a sound basis for any attempt to design an optimal computational system. The focus of this study is the efficiency and the effectiveness with which high-salience subsets of features can be identified in the context of ill-understood and potentially noisy real-world data. Our two simple approaches, weight clamping using a neural network and feature ranking using a decision tree, generally provide a good, consistent ordering of features. In addition, linear correlation often works well.

Engineering artificial intelligence software

Artificial Intelligence (AI) software is a reality, but only for limited classes of problems. In general, AI problems are significantly different from those of conventional software engineering. The differences suggest a different program development methodology for AI problems: one that does not readily yield programs with the desiderata of practical software (reliability, robustness, etc.). In addition, the problem of machine learning must be solved (to some degree) before the full potential of AI can be realized, but the resultant self-adaptive software is likely to further aggravate the software crisis. Realization of the full potential of AI in practical software awaits some prerequisite breakthroughs in both basic AI problems and an appropriate AI software development methodology.