Stephen J. Verzi

Exemplar-based pattern recognition via semi-supervised learning

The focus of this paper is semi-supervised learning in the context of pattern recognition. Semi-supervised learning (SSL) refers to the semi-supervised construction of clusters during the training phase of exemplar-based classifiers. Using artificially generated data sets we present experimental results of classifiers that follow the SSL paradigm and we show that, especially for difficult pattern recognition problems featuring high class overlap, for exemplar-based classifiers implementing SSL i) the generalization performance improves, while ii) the number of necessary exemplars decreases significantly, when compared to the original versions of the classifiers.

Boosted ARTMAP: Modifications to fuzzy ARTMAP motivated by boosting theory

In this paper, several modifications to the Fuzzy ARTMAP neural network architecture are proposed for conducting classification in complex, possibly noisy, environments. The goal of these modifications is to improve upon the generalization performance of Fuzzy ART-based neural networks, such as Fuzzy ARTMAP, in these situations. One of the major difficulties of employing Fuzzy ARTMAP on such learning problems involves over-fitting of the training data. Structural risk minimization is a machine-learning framework that addresses the issue of over-fitting by providing a backbone for analysis as well as an impetus for the design of better learning algorithms. The theory of structural risk minimization reveals a trade-off between training error and classifier complexity in reducing generalization error, which will be exploited in the learning algorithms proposed in this paper. Boosted ART extends Fuzzy ART by allowing the spatial extent of each cluster formed to be adjusted independently. Boosted ARTMAP generalizes upon Fuzzy ARTMAP by allowing non-zero training error in an effort to reduce the hypothesis complexity and hence improve overall generalization performance. Although Boosted ARTMAP is strictly speaking not a boosting algorithm, the changes it encompasses were motivated by the goals that one strives to achieve when employing boosting. Boosted ARTMAP is an on-line learner, it does not require excessive parameter tuning to operate, and it reduces precisely to Fuzzy ARTMAP for particular parameter values. Another architecture described in this paper is Structural Boosted ARTMAP, which uses both Boosted ART and Boosted ARTMAP to perform structural risk minimization learning. Structural Boosted ARTMAP will allow comparison of the capabilities of off-line versus on-line learning as well as empirical risk minimization versus structural risk minimization using Fuzzy ARTMAP-based neural network architectures. Both empirical and theoretical results are presented to enhance the understanding of these architectures.

Evaluating Moving Target Defense with PLADD

This project evaluates the effectiveness of moving target defense (MTD) techniques using a new game we have designed, called PLADD, inspired by the game FlipIt [28]. PLADD extends FlipIt by incorporating what we believe are key MTD concepts. We have analyzed PLADD and proven the existence of a defender strategy that pushes a rational attacker out of the game, demonstrated how limited the strategies available to an attacker are in PLADD, and derived analytic expressions for the expected utility of the game’s players in multiple game variants. We have created an algorithm for finding a defender’s optimal PLADD strategy. We show that in the special case of achieving deterrence in PLADD, MTD is not always cost effective and that its optimal deployment may shift abruptly from not using MTD at all to using it as aggressively as possible. We believe our effort provides basic, fundamental insights into the use of MTD, but conclude that a truly practical analysis requires model selection and calibration based on real scenarios and empirical data. We propose several avenues for further inquiry, including (1) agents with adaptive capabilities more reflective of real world adversaries, (2) the presence of multiple, heterogeneous adversaries, (3) computational game theory-based approaches such as coevolution to allow scaling to the real world beyond the limitations of analytical analysis and classical game theory, (4) mapping the game to real-world scenarios, (5) taking player risk into account when designing a strategy (in addition to expected payoff), (6) improving our understanding of the dynamic nature of MTD-inspired games by using a martingale representation, defensive forecasting, and techniques from signal processing, and (7) using adversarial games to develop inherently resilient cyber systems.

Foundations to the unified psycho-cognitive engine.

This document outlines the key features of the SNL psychological engine. The engine is designed to be a generic presentation of cognitive entities interacting among themselves and with the external world. The engine combines the most accepted theories of behavioral psychology with those of behavioral economics to produce a unified simulation of human response from stimuli through executed behavior. The engine explicitly recognizes emotive and reasoned contributions to behavior and simulates the dynamics associated with cue processing, learning, and choice selection. Most importantly, the model parameterization can come from available media or survey information, as well subject-matter-expert information. The framework design allows the use of uncertainty quantification and sensitivity analysis to manage confidence in using the analysis results for intervention decisions.

Game theoretic mechanism design applied to machine learning classification

The field of machine learning strives to develop algorithms that, through learning, lead to generalization; that is, the ability of a machine to perform a task that it was not explicitly trained for. Numerous approaches have been developed ranging from neural network models striving to replicate neurophysiology to more abstract mathematical manipulations which identify numerical similarities. Nevertheless a common theme amongst the varied approaches is that learning techniques incorporate a strategic component to try and yield the best possible decision or classification. The mathematics of game theory formally analyzes strategic interactions between competing players and is consequently quite appropriate to apply to the field of machine learning with potential descriptive as well as functional insights. Furthermore, game theoretic mechanism design seeks to develop a framework to achieve a desired outcome, and as such is applicable for defining a paradigm capable of performing classification. In this work we present a game theoretic chip-fire classifier which as an iterated game is able to perform pattern classification.

Adult Neurogenesis: Implications on Human And Computational Decision Making

Adult neurogenesis is the incorporation of new neurons into established, functioning neural circuits. Current theoretical work in the neurogenesis field has suggested that new neurons are of greatest importance in the encoding of new memories, particularly in the ability to fully capture features which are entirely novel or being experienced in a unique way. We present two models of neurogenesis (a spiking, biologically realistic model as well as a basic growing feedforward model) to investigate possible functional implications. We use an information theoretic computational complexity measure to quantitatively analyze the information content encoded with and without neurogenesis in our spiking model. And neural encoding capacity (as a function of neuron maturation) is examined in our simple feedforward network. Finally, we discuss potential functional implications for neurogenesis in high risk environments.

Generalization performance of ARTMAP-based networks in structural risk minimization framework

Many techniques have been proposed for improving the generalization performance of fuzzy ARTMAP. We present a study of these architectures in the framework of structural risk minimization and computational learning theory. Fuzzy ARTMAP training uses on-line learning, has proven convergence results, and has relatively few parameters to deal with. Empirical risk minimization is employed by fuzzy ARTMAP during its training phase. One weakness of fuzzy ARTMAP concerns over-training on noisy training data sets or naturally overlapping training classes of data. Most of these proposed techniques attempt to address this issue, in different ways, either directly or indirectly. In this paper we will present a summary of how some of these architectures achieve success as learning algorithms.

Fuzzy Set Methods for Assessing Uncertainty in the Modeling and Control of Space Structures

Aerospace institutions such as NASA and the U.S. Air Force have long been interested in the development of methods for evaluating the predictive accuracy of structural dynamic models. This interest is due to the fact that mathematical models are used to evaluate the structural integrity of all aircraft and spacecraft prior to flight. Space structures are often too large and too weak to be tested fully assembled in a ground test laboratory. The predictive accuracy of a model depends on the nature and extent of its experimental verification. The further the test conditions depart from in-service conditions, the less accurate the model is likely be. The best method for quantitatively evaluating the predictive accuracy of a model is to make direct measurements under simulated service conditions. Unfortunately, this method is expensive and fraught with problems in achieving service conditions on earth. This article presents progress made in the combined use of several methods to evaluate the accuracy of dynamic models of large space structures using numerical simulation. Some of these methods involve the theory of fuzzy sets. The fuzzy set methods are shown to be effective and computationally efficient as tools for bounding the range of possible responses, segregating important modal responses from those having less effect on predicted response, reduction of uncertainty in plant models in a control-structure interaction context, and providing the only plausible means of uncertainty prediction at the poles and zeros of the frequency response spectra. The article illustrates these notions with some numerical examples.

Resource Requirements Planning for Hospitals Treating Serious Infectious Disease Cases

This report presents a mathematical model of the way in which a hospital uses a variety of resources, utilities and consumables to provide care to a set of in-patients, and how that hospital might adapt to provide treatment to a few patients with a serious infectious disease, like the Ebola virus. The intended purpose of the model is to support requirements planning studies, so that hospitals may be better prepared for situations that are likely to strain their available resources. The current model is a prototype designed to present the basic structural elements of a requirements planning analysis. Some simple illustrati ve experiments establish the mo dels general capabilities. With additional inve stment in model enhancement a nd calibration, this prototype could be developed into a useful planning tool for ho spital administrators and health care policy makers.

Augmented cognition tool for rapid military decision making.

This report describes the laboratory directed research and development work to model relevant areas of the brain that associate multi-modal information for long-term storage for the purpose of creating a more effective, and more automated, association mechanism to support rapid decision making. Using the biology and functionality of the hippocampus as an analogy or inspiration, we have developed an artificial neural network architecture to associate k-tuples (paired associates) of multimodal input records. The architecture is composed of coupled unimodal self-organizing neural modules that learn generalizations of unimodal components of the input record. Cross modal associations, stored as a higher-order tensor, are learned incrementally as these generalizations form. Graph algorithms are then applied to the tensor to extract multi-modal association networks formed during learning. Doing so yields a novel approach to data mining for knowledge discovery. This report describes the neurobiological inspiration, architecture, and operational characteristics of our model, and also provides a real world terrorist network example to illustrate the models functionality.