James T. Graham

Predictive Network Anomaly Detection and Visualization

Various approaches have been developed for quantifying and displaying network traffic information for determining network status and in detecting anomalies. Although many of these methods are effective, they rely on the collection of long-term network statistics. Here, we present an approach that uses short-term observations of network features and their respective time averaged entropies. Acute changes are localized in network feature space using adaptive Wiener filtering and auto-regressive moving average modeling. The color-enhanced datagram is designed to allow a network engineer to quickly capture and visually comprehend at a glance the statistical characteristics of a network anomaly. First, average entropy for each feature is calculated for every second of observation. Then, the resultant short-term measurement is subjected to first- and second-order time averaging statistics. These measurements are the basis of a novel approach to anomaly estimation based on the well-known Fisher linear discriminant (FLD). Average port, high port, server ports, and peered ports are some of the network features used for stochastic clustering and filtering. We empirically determine that these network features obey Gaussian-like distributions. The proposed algorithm is tested on real-time network traffic data from Ohio Universitys main Internet connection. Experimentation has shown that the presented FLD-based scheme is accurate in identifying anomalies in network feature space, in localizing anomalies in network traffic flow, and in helping network engineers to prevent potential hazards. Furthermore, its performance is highly effective in providing a colorized visualization chart to network analysts in the presence of bursty network traffic.

Motivated learning for the development of autonomous systems

A new machine learning approach known as motivated learning (ML) is presented in this work. Motivated learning drives a machine to develop abstract motivations and choose its own goals. ML also provides a self-organizing system that controls a machines behavior based on competition between dynamically-changing pain signals. This provides an interplay of externally driven and internally generated control signals. It is demonstrated that ML not only yields a more sophisticated learning mechanism and system of values than reinforcement learning (RL), but is also more efficient in learning complex relations and delivers better performance than RL in dynamically-changing environments. In addition, this paper shows the basic neural network structures used to create abstract motivations, higher level goals, and subgoals. Finally, simulation results show comparisons between ML and RL in environments of gradually increasing sophistication and levels of difficulty.

Opportunistic Behavior in Motivated Learning Agents

This paper focuses on the novel motivated learning (ML) scheme and opportunistic behavior of an intelligent agent. It extends previously developed ML to opportunistic behavior in a multitask situation. Our paper describes the virtual world implementation of autonomous opportunistic agents learning in a dynamically changing environment, creating abstract goals, and taking advantage of arising opportunities to improve their performance. An opportunistic agent achieves better results than an agent based on ML only. It does so by minimizing the average value of all need signals rather than a dominating need. This paper applies to the design of autonomous embodied systems (robots) learning in real-time how to operate in a complex environment.

Anomaly prediction in network traffic using adaptive Wiener filtering and ARMA modeling

Fast and efficient detection of anomalies is essential for maintaining a robust and secure network. This research presents a method of anomaly detection based on adaptive Wiener filtering of noise followed by ARMA modeling of network flow data. We dynamically calculate noise and traffic signal statistics using network-monitoring metrics for traffic features such as average port, high port, server ports, and peered ports. The underlying approach is tested on near-real-time Internet traffic in the wide-area network (WAN) of Ohio University. The average port feature is determined to be the most informative measure in the estimation process. High port, server ports, and peered ports are used for confirmation of the anomaly detection result. We empirically determine that most of the network features obey Gaussian-like distributions. Experiments reveal that the method is highly effective in predicting anomalies in network traffic flow and preventing any hazard that they may cause.

MLECOG: Motivated Learning Embodied Cognitive Architecture

This paper presents a new cognitive architecture and its major functional blocks. It describes how the new architecture is related to major trends in cognitive architectures that move toward greater autonomy, motivations, and the creation of goals. After a brief characterization of existing cognitive architectures, particularly those that share similarities with our own, the desired architectural requirements for embodied cognitive systems are spelled out. The proposed cognitive architecture is then presented on a functional level that describes how its functional blocks interact to self-organize and to refine their operations. Selected implementation issues are also discussed. Finally, simulation results of an implemented simplified version of the proposed system are discussed. The simulation results show that the current implementation of the motivated learning embodied cognitive architecture system is capable of maintaining itself in a dynamic environment.

Anomaly detection and visualization using Fisher Discriminant clustering of network entropy

Entropy has been widely used to quantify information for display and examination in determining network status and in detecting anomalies. Although entropy-based methods are effective, they rely on long-term network statistics. Here, we propose an approach that deduces short term observations of network features and their respective time averaged entropies. Acute changes are detected in network feature space and depicted in a visually compact information graph. First, average entropy for each feature is calculated for every second of observation. Then, the resultant short-term information measurement is subjected to first- and second-order time averaging statistics. These time-varying statistics are used as the basis of a novel approach to anomaly estimation based on the well-known Fisher linear discriminant (FLD). This process then initiates stochastic clustering to identify the exact time of the security incident or attack on the network. The proposed method is tested on real-time network traffic data collected from Ohio Universitypsilas main Internet connection. Experimentation has shown that the presented FLD based method is accurate in identifying anomalies in network feature space. Furthermore, itpsilas performance is highly robust in the presence of bursty network traffic and it is able to detect network anomalies such as BotNet, worm outbreaks, and denial of service attacks.

Simulation of a Motivated Learning Agent

In this paper we discuss how to design a simple motivated learning agent with symbolic I/O using a simulation environment within the NeoAxis game engine. The purpose of this work is to explore autonomous development of motivations and memory of agents in a virtual environment. The approach we took should speed up the development process, bypassing the need to create a physical embodied agent as well as reducing the learning effort. By rendering low-level motor actions such as grasping or walking into symbolic commands we remove the need to learn elementary motions. Instead, we use several basic primitive motor procedures, which can form more complex procedures. Furthermore, by simulating the agent’s environment, we both improve and simplify the learning process. There are a few adaptive learning variables associated with both the agent and its environment, and learning takes less time, than it would in a more complex real world environment.

A hybrid self-organizing Neural Gas based network

This paper examines the neural gas networks proposed by Martinetz and Schulten and Fritzke in an effort to create a more biologically plausible hybrid version. The hybrid algorithm proposed in this work retains most of the advantages of the Growing Neural Gas (GNG) algorithm while adapting a reduced parameter and more biologically plausible design. It retains the ability to place nodes where needed, as in the GNG algorithm, without actually having to introduce new nodes. Also, by removing the weight and error adjusting parameters, the guesswork required to determine parameters is eliminated. When compared to Fritzkepsilas algorithm, the hybrid algorithm performs admirably in terms of the quality of results it is slightly slower due to the greater computational overhead. However, it is more biologically feasible and somewhat more flexible due to its hybrid nature and lack of reliance on adjustment parameters.

Opportunistic motivated learning agents

This paper presents an extension of the Motivated Learning model that includes environment masking, and opportunistic behavior of the motivated learning agent. Environment masking improves an agents ability to learn by helping to filter out distractions, and the addition of a more complex environment increases the simulations realism. If conditions call for it opportunistic behavior allows an agent to deviate from the dominant task to perform a less important but rewarding action. Numerical simulations were performed using Matlab and the implementation of a graphical simulation based on the OGRE engine is in progress. Simulation results show good performance and numerical stability of the attained solution.

A comparative study between motivated learning and reinforcement learning

This paper analyzes advanced reinforcement learning techniques and compares some of them to motivated learning. Motivated learning is briefly discussed indicating its relation to reinforcement learning. A black box scenario for comparative analysis of learning efficiency in autonomous agents is developed and described. This is used to analyze selected algorithms. Reported results demonstrate that in the selected category of problems, motivated learning outperformed all reinforcement learning algorithms we compared with.