Steven A. Harp

Imputation of Missing Data in Industrial Databases

A limiting factor for the application of IDA methods in many domains is the incompleteness of data repositories. Many records have fields that are not filled in, especially, when data entry is manual. In addition, a significant fraction of the entries can be erroneous and there may be no alternative but to discard these records. But every cell in a database is not an independent datum. Statistical relationships will constrain and, often determine, missing values. Data imputation, the filling in of missing values for partially missing data, can thus be an invaluable first step in many IDA projects. New imputation methods that can handle the large-scale problems and large-scale sparsity of industrial databases are needed. To illustrate the incomplete database problem, we analyze one database with instrumentation maintenance and test records for an industrial process. Despite regulatory requirements for process data collection, this database is less than 50% complete. Next, we discuss possible solutions to the missing data problem. Several approaches to imputation are noted and classified into two categories: data-driven and model-based. We then describe two machine-learning-based approaches that we have worked with. These build upon well-known algorithms: AutoClass and C4.5. Several experiments are designed, all using the maintenance database as a common test-bed but with various data splits and algorithmic variations. Results are generally positive with up to 80% accuracies of imputation. We conclude the paper by outlining some considerations in selecting imputation methods, and by discussing applications of data imputation for intelligent data analysis.

Self–organization with partial data

We show how the kohonen self-organizing feature map model can be extended so that partial training data can be utilized. Given input stimuli in which values for some elements or features are absent, the match computation and the weight updates are performed in the input subspace defined by the available values. Three examples, including an application to student modelling for intelligent tutoring systems in which data is inherently incomplete, demonstrate the effectiveness of the extension.

Information modeling for intrusion report aggregation

The paper describes the SCYLLARUS approach to fusing reports from multiple intrusion detection systems (ID-Ses) to provide an overall approach to intrusion situation awareness. The overall view provided by SCYLLARUS centers around the sites security goals, aggregating large numbers of individual IDS reports based on their impact. The overall view reduces information overload by aggregating multiple IDS reports in a rep-down view; and by reducing false positives by weighing evidence provided by multiple ID-Ses and other information sources. Unlike previous efforts in this area, SCYLLARUS is centered around its intrusion reference model (IRM). The SCYLLARUS IRM contains both dynamic and static (configuration) information. A network entity/relationship database (NERD), providing information about the sites hardware and software; a security goal database, describing the sites objectives and security policy; and an event dictionary, describing important events, both intrusions and benign; comprise the static portion of the IRM. The set of IDS reports; the events SCYLLARUS hypothesizes to explain them; and the resulting judgment of the state of site security goals comprise the dynamic part of the IRM.

Modeling student knowledge with self-organizing feature maps

The paper describes a novel application of neural networks to model the behavior of students in the context of an intelligent tutoring system. Self-organizing feature maps are used to capture the possible states of student knowledge from an existing test database. The trained network implements a universal student knowledge model that is compatible with knowledge space theory approaches to student assessment and computer aided instruction. The student model can be applied to rapidly assess the knowledge of any given student, and chart a path from lower to higher states of expertise. The authors illustrate the concept on an aircraft fuel management domain, demonstrating its noise-tolerance and insensitivity to feature map parameter values. An approach to determining the correct feature map size is also described. >

Textual data mining of service center call records

In this project, w e dev eloped a technique for extracting useful information from databases that contain both xedformat and free-text elds. The present state of the art in data mining is a schism betw een tec hniques that handle only xed-format data (pattern recognition, classi cation algorithms from machine learning), and techniques designed for free-form text (information retrieval). Advanced knowledge disco very technologies ha ve been developed in both research areas, but systems that can categorize or cluster records containing both kinds of data are still lacking. Speci cally, we examined database records from a Honeywell service cen ter to extract information about the expected cost of di erent kinds of service requests. Our goal was to test the h ypothesis that incorporating information from free-text elds would provide a better categorization of these records; in this case, better predictions of the cost of the service call. In our w ork, we have integrated feature extraction and clustering techniques from information retrieval with classi cation algorithms from machine learning in order to categorize the hybrid elds. Our preliminary results suggested that incorporating free-form text could potentially induce better classi cation models.

Qualitative user aiding for alarm management (QUALM): an integrated demonstration of emerging technologies for aiding process control operators

Managing abnormal situations in industrial processes has become increasingly challenging due to the increased sophistication of both the process as well as the automated control system. Operators can benefit significantly from tools that assist in abnormal situation management. Some critical technologies that can help operators include: user intent recognition, diagnosis and advanced graphical user interface. While each of these technologies independently have been proven to be valuable, enhanced benefits can be reaped by integrating all these into a single framework. In this work, we describe a framework for integrating these technologies to provide the operator with a single system. A real-time simulation of a unit in a chemical plant is used to test the value of the integrated framework. This effort has helped us identify functional requirements of the various modules and the mechanism in which they interact.<<ETX>>

Trapping Malicious Insiders in the SPDR Web

The insider threat has assumed increasing importance as our dependence on critical cyber information infrastructure has increased. In this paper we describe an approach for thwarting and attributing insider attacks. The Sense, Prepare, Detect, and React (SPDR) approach utilizes both a highly intelligent software reasoning system to anticipate, recognize, respond to, and attribute attacks as well as a widely distributed set of hardware-based sensor-effectors to provide alerts used by the reasoning system and to implement responses as directed by it. Using hardware sensor-effectors greatly reduces the risk that a savvy malicious insider can bypass or cripple the system’s monitoring and control capabilities. In this paper we describe the prototype SPDR system and the results of its successful evaluation by an independent, DARPA-sponsored Red Team. We conclude with thoughts on possible SPDR enhancements and further research.

Frankencode: Creating Diverse Programs Using Code Clones

In this paper, we present an approach to detecting novel cyber attacks though a form of program diversification, similar to the use of n-version programming for fault tolerant systems. Building on extensive previous and ongoing work by others on the use of code clones in a wide variety of areas, our Functionally Equivalent Variants using Information Synchronization (FEVIS) system automatically generates program variants to berun in parallel, seeking to detect attacks through divergence in behavior. Unlike approaches to diversification that only change program memory layout and behavior, FEVIS can detect attacks exploiting vulnerabilities in execution timing, string processing, and other logic errors. We are in the early stages of research and development for this approach, but have made sufficient progress to provide a proof of concept and some lessons learned. In this paper we describe FEVIS and its application to diversifying an open-source webserver, with results on several different example classes of attack which FEVIS will detect.

An architecture for scalable network defense

We describe a novel architecture for network defense designed for scaling to very high data rates (100 Gb/s) and very large user populations. Scaling requires both efficient attack detection algorithms as well as appropriate an execution environment. Our architecture considers the time budget of traffic data extraction and algorithmic processing, provides a suite of detection algorithms”each designed to present different and complementary views of the data—that generate many “traffic events,” and reduces false positives by correlating these traffic events into benign or malicious hypotheses.

Improving self defense by learning from limited experience

Prevalence of new attacks or attack variants presents an interesting challenge for autonomic cyber-defense: how does the autonomic defense mechanism learn from previous failures, acquiring immunity with experience, and do so as rapidly as possible. In the limiting case, only a single a single observed failure may be available for learning.