Guangzhi Qu

A new dependency and correlation analysis for features

The quality of the data being analyzed is a critical factor that affects the accuracy of data mining algorithms. There are two important aspects of the data quality, one is relevance and the other is data redundancy. The inclusion of irrelevant and redundant features in the data mining model results in poor predictions and high computational overhead. This paper presents an efficient method concerning both the relevance of the features and the pairwise features correlation in order to improve the prediction and accuracy of our data mining algorithm. We introduce a new feature correlation metric Q/sub Y/(X/sub i/,X/sub j/) and feature subset merit measure e(S) to quantify the relevance and the correlation among features with respect to a desired data mining task (e.g., detection of an abnormal behavior in a network service due to network attacks). Our approach takes into consideration not only the dependency among the features, but also their dependency with respect to a given data mining task. Our analysis shows that the correlation relationship among features depends on the decision task and, thus, they display different behaviors as we change the decision task. We applied our data mining approach to network security and validated it using the DARPA KDD99 benchmark data set. Our results show that, using the new decision dependent correlation metric, we can efficiently detect rare network attacks such as User to Root (U2R) and Remote to Local (R2L) attacks. The best reported detection rates for U2R and R2L on the KDD99 data sets were 13.2 percent and 8.4 percent with 0.5 percent false alarm, respectively. For U2R attacks, our approach can achieve a 92.5 percent detection rate with a false alarm of 0.7587 percent. For R2L attacks, our approach can achieve a 92.47 percent detection rate with a false alarm of 8.35 percent.

Impact analysis of faults and attacks in large-scale networks

Monitoring and quantifying component behavior is key to, making networks reliable and robust. The agent-based architecture presented here continuously monitors network vulnerability metrics providing new ways to measure the impact of faults and attacks.

An efficient network intrusion detection method based on information theory and genetic algorithm

The Internet has been growing at an amazing rate and concurrent with the growth, the vulnerability of the Internet is also increasing. Though the Internet has been designed to withstand various forms of failure, the intrusion tools and attacks are becoming increasingly sophisticated, exposing the Internet to new threats. To make networked systems reliable and robust it becomes highly essential to develop on-line monitoring, analysis and quantification of the behavior of networks under a wide range of attacks and to recover from these attacks. In this paper, we present a hybrid method based on information theory and genetic algorithm to detect network attacks. Our approach uses information theory to filter the traffic data and thus reduce the complexity. We use a linear structure rule to classify the network behaviors into normal and abnormal behaviors. We apply our approach to the kdd99 benchmark dataset and obtain high detection rate of 99.25% as well as low false alarm rate of 1.66%.

Multi-Level Intrusion Detection System (ML-IDS)

As the deployment of network-centric systems increases, network attacks are proportionally increasing in intensity as well as complexity. Attack detection techniques can be broadly classified as being signature-based, classification-based, or anomaly-based. In this paper we present a multi level intrusion detection system (ML-IDS) that uses autonomic computing to automate the control and management of ML-IDS. This automation allows ML-IDS to detect network attacks and proactively protect against them. ML-IDS inspects and analyzes network traffic using three levels of granularities (traffic flow, packet header, and payload), and employs an efficient fusion decision algorithm to improve the overall detection rate and minimize the occurrence of false alarms. We have individually evaluated each of our approaches against a wide range of network attacks, and then compared the results of these approaches with the results of the combined decision fusion algorithm.

Multivariate statistical analysis for network attacks detection

Summary form only given. Detection and self-protection against viruses, worms, and network attacks is urgently needed to protect network systems and their applications from catastrophic failures. Once a network component is infected by viruses, worms, or became a target of network attacks, its operational state shifts from normal to abnormal state. Online monitoring mechanism can collect important aspects of network traffic and host data (CPU utilization, memory usage, etc.), that can be effectively used to detect abnormal behaviors caused by attacks. In this paper, we develop an online multivariate analysis algorithm to analyze the behaviors of system resources and network protocols in order to proactively detect network attacks. We have validated an algorithm and showed how it can proactively detect accurately well-known attacks such as distributed denial of service, SQL slammer worm, and email spam attacks.

Quality-of-protection (QoP)-an online monitoring and self-protection mechanism

With increasing faults and attacks on the Internet infrastructure, there is an impending need to provide automatic techniques to detect and mitigate the impact of attacks on network services. Denial-of-service attacks have been successful in denying legitimate traffic access to its required resources because existing routing protocols treat the attacking traffic equally as any normal traffic. This paper presents a proactive network defense framework that can be integrated with existing quality-of-service (QoS) protocols to provide differentiated services to network traffic flows based on their distance from the normal behavior. We introduce a new metric that we refer to as abnormality distance (AD) metric that can be used to classify traffic into normal, probable normal, probable abnormal (suspicious traffic), and abnormal (attacking traffic). The AD metric can then be used in conjunction with any QoS protocol to give high priority to normal traffic and lower priority to abnormal traffic. We demonstrate through several examples, how our approach can dynamically detect attacks, quantify their impact, and how to reduce the impacts and recover from them.

Fast algorithms to evaluate collaborative filtering recommender systems

Before deploying a recommender system, its performance must be measured and understood. So evaluation is an integral part of the process to design and implement recommender systems. In collaborative filtering, there are many metrics for evaluating recommender systems. Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) are among the most important and representative ones. To calculate MAE/RMSE, predicted ratings are compared with their corresponding true ratings. To predict item ratings, similarities between active users and their candidate neighbors need to be calculated. The complexity for the traditional and naive similarity calculation corresponding to user u and user v is quadratic in the number of items rated by u and v. In this paper, we explore the mathematical regularities underlying the similarity formulas, introduce a novel data structure, and design linear time algorithms to calculate the similarities. Such complexity improvement shortens the evaluation time and will finally contribute to increasing the efficiency of design and development of recommender systems. Experimental results confirm the claim.

Self-Configuration of Network Security

The proliferation of networked systems and services along with their exponential growth in complexity and size has increased the control and management complexity of such systems and services by several orders of magnitude. As a result, management tools have failed to cope with and handle the complexity, dynamism, and coordination among network attacks. In this paper, we present a self-configuration approach to control and manage the security mechanisms of large scale networks. Self-configuration enables the system to automatically configure security system and change the configuration of its resources and their operational policies at runtime in order to manage the system security. Our self-configuration approach is implemented using two software modules: component management interface (CMI) to specify the configuration and operational policies associated with each component that can be a hardware resource or a software component; and component runtime manager (CRM) that manages the component operations using the policies defined in CMI. We have used the self-configuration framework to experiment with and evaluate different mechanisms and strategies to detect and protect against a wide range of network attacks.

Online monitoring and analysis for self-protection against network attacks

In this paper, we present an online monitoring and analysis framework to achieve self-protection against a wide range of network attacks. Our approach uses the software agents to online monitor several attributes to characterize the state of any network or computing resource as normal, uncertain, or abnormal. The software agents execute the appropriate recovery mechanisms once they determine that a service, and/or a network device is operating abnormally. We have developed a test bed to demonstrate and validate our approach to protect against several well-known attacks.

Complex networks properties analysis for mobile ad hoc networks

Recently, research on complex network theory and applications draws a lot of attention in both academy and industry. In mobile ad hoc networks (MANETs) area of research, a critical issue is to design the most effective topology for given problems. It is natural and significant to consider complex networks topology when optimising the MANET topology. Current works usually transform MANET or sensor network topologies into either small-world or scale-free. However, some fundamental problems remain unsolved. Specifically, what are the average shortest path length, degree distribution and clustering characteristics of MANETs? Do MANETs have small-world effect and scale-free property? In this work, the authors introduce complex networks theory into the context of MANET topology and study complex network properties of the MANETs to answer the above questions. The authors have theoretically analysed the degree distribution and clustering coefficient of MANETs and proposed approach to computing them. The degree distribution and clustering coefficient of MANETs are theoretically deduced from node space probability distribution on different mobility models (including but not limited to random waypoint model). Simulation results on average shortest path length, clustering coefficient and degree distribution show that in most cases MANETs do not have the small-world effect and scale-free property.