Zhuowei Li

Theoretical basis for intrusion detection

Intrusion detection has become an indispensable defense line in the information security infrastructure. However, every intrusion detection approach has been limited by their problems: signature-based intrusion detection can identify the known intrusions but cannot detect the novel intrusions, anomaly-based intrusion detection has the potential to detect all intrusions but has the limitation of a higher false alarm rate. For this reason, most existing intrusion detection techniques have not met the requirements for practical deployment. In this paper, the authors proposed a theoretical basis for intrusion detection to argue about their principles and to analyze the existing problems for intrusion detection in a quantified manner. The root causes of these problems are identified as model inaccuracy and model incompleteness as well as the distinguishability lack in the features utilized. In addition, it is also found that static analysis (Wagner, et al., 2001), with a properly selected feature vector, is a promising intrusion detection technique in principle because it can avoid the quality issue of its behavior models.

USAID: unifying signature-based and anomaly-based intrusion detection

Most intrusion detection techniques suffer from either an inability to detect unknown intrusions, or unacceptably high false alarm rates. However, there lacks a general basis to analyze and find solutions to these problems. In this paper, we propose such a theoretical basis for intrusion detection, which makes it possible to systematically express and analyze the detection performance metrics such as the detection rate and false alarm rate in a quantified manner. Most importantly, the insights gained from the basis lead to the proposal for a new intrusion detection technique – USAID. USAID attempts to exploit the advantages of both techniques, and overcome their respective shortcomings. The experimental results show that USAID can achieve uniform level of efficiency to detect both known (99.78%) and new intrusions (98.18%), with a significantly reduced false alarm rate (1.45%). Most significantly, the performance of USAID is superior to all the participants in KDD99 if the anomalies detected by USAID can be categorized correctly.

Model generalization and its implications on intrusion detection

To make up for the incompleteness of the known behaviors of a computing resource, model generalization is utilized to infer more behaviors in the behavior model besides the known behaviors. In principle, model generalization can improve the detection rate but may also degrade the detection performance. Therefore, the relation between model generalization and detection performance is critical for intrusion detection. However, most of past research only evaluates the overall efficiency of an intrusion detection technique via detection rate and false alarm/positive rate, rather than the usefulness of model generalization for intrusion detection. In this paper, we try to do such evaluation, and then to find the implications of model generalization on intrusion detection. Within our proposed methodology, model generalization can be achieved in three levels. In this paper, we evaluate the first level model generalization. The experimental results show that the first level model generalization is useful mostly to enhance the detection performance of intrusion detection. However, its implications for intrusion detection are different with respect to different detection techniques. Our studies show that in general, though it is useful to generalize the normal behavior model so that more normal behaviors can be identified as such, the same is not advisable for the intrusive behavior model. Therefore, the intrusion signatures should be built compactly without first level generalization.

Visualizing and identifying intrusion context from system calls trace

Anomaly-based intrusion detection (AID) techniques are useful for detecting novel intrusions without known signatures. However, AID techniques suffer from higher false alarm rate compared to signature-based intrusion detection techniques. In this paper, the concept of intrusion context identification is introduced to address the problem. The identification of the intrusion context can help to significantly enhance the detection rate and lower the false alarm rate of AID techniques. To evaluate the effectiveness of the concept, a simple but representative scheme for intrusion context identification is proposed, in which the anomalies in the intrusive datasets are visualized first, and then the intrusion contexts are identified from the visualized anomalies. The experimental results show that using the scheme, the intrusion contexts can be visualized and extracted from the audit trails correctly. In addition, as an application of the visualized anomalies, an implicit design drawback in t-stide is found after careful analysis. Finally, based on the identified intrusion context and the efficiency comparison, several findings are made which can offer useful insights and benefit future research on AID techniques.

Analyzing and evaluating dynamics in stide performance for intrusion detection

Anomaly-based intrusion detection (AID) techniques are useful for detecting novel intrusions into computing resources. One of simple but typical AID detectors proposed to date is stide, which is based on analysis of system call sequences. In this paper, we present a detailed formal framework to analyze, understand and improve the performance of stide and similar AID techniques. Several important properties of stide-like detectors are established through formal theorems, and validated by carefully conducted experiments using test datasets. Finally, the framework is utilized to reduce the cost of developing AID detectors by identifying the critical sections in the training dataset.

The Utility of Partial Knowledge in Behavior Models: an Evaluation for Intrusion Detection

To enlarge the detection capability of an incomplete behavior model, model generalization is necessary to make every behavior signature identify more behavior instances. In this paper, based on a general intrusion detection framework, M out of N features in a behavior signature are utilized to detect the behaviors (M ≤ N) instead of using all N features. This is because M of N features in a signature can generalize the behavior model to incorporate unknown behaviors, which are useful to detect novel intrusions outside the known behavior model. However, the preliminary experimental results show that all features of any signature should be fully utilized for intrusion detection instead of M features in it. This is because the M of N features scheme will make the behavior identification capability of the behavior model lost by detecting most behaviors as anomalies or alarms.

M of N features vs. intrusion detection

In order to complement the incomplete training audit trails, model generalization is always utilized to infer more unknown knowledge for intrusion detection. Thus, it is important to evaluate model generalization with respect to the detection performance of intrusion detection. In this paper, based on a general intrusion detection methodology, M out of N features in a behavior signature are utilized to detect the behaviors (M ≤ N) instead of using all N features. This is because M of N features in a signature can generalize the behavior model to incorporate unknown behaviors, which are useful to detect novel intrusions outside the known behavior model. However, the preliminary experimental results show that all features of any signature should be fully utilized for intrusion detection instead of M features in it. This is because the M of N features scheme will make the behavior identification capability of the behavior model lost by detecting most behaviors as ‘anomalies.

Model redundancy vs. intrusion detection

A major problem faced by intrusion detection is the intensive computation in the detection phase, and a possible solution is to reduce model redundancy, and thus economize the detection computation. However, the existing literature lacks any formal evaluation of the significance of model redundancy for intrusion detection. In this paper, we try to do such an evaluation. First, in a general intrusion detection methodology, the model redundancy in the behavior model can be reduced using feature ranking and the proposed concept of ‘variable-length signature. Then, the detection performance of the behavior model before and after model redundancy is compared. The preliminary experimental results show that the model redundancy in the behavior model is useful to detect novel intrusions, but the model redundancy due to the overlapping distinguishability among features is insignificant for intrusion detection.

Towards automatic assembly of privacy-preserved intrusion signatures

Intrusion signatures are used to detect and/or prevent fast-spreading worms or exploits, and usually, constructing these signatures is an automatic process without human intervention for the sake of speed. In principle, the automatic signature construction process can produce not only true-positive intrusion signatures but also false-positive ones, the latter of which poses a grave problem because they can be misused to disclose privacy information. Manual signature checking (for a whitelist) can solve the problem, but it slows down the reaction time for an attack dramatically. In this paper, we propose a mechanism to generate signatures automatically while preserving the privacy information. Essentially, we transform the original feature values within an audit trail instance into feature ranges, and then use these feature ranges to construct a privacy-preserved intrusion signature. Our current focus is on the methods constructing feature ranges, and for this purpose, several methods are proposed to discover feature ranges. The experimental results are quite encouraging: the transformation from values to ranges leads not only to the preservation of privacy but also to the enhancement of the detection performance.