Bin Zhang

PCA-subspace method — Is it good enough for network-wide anomaly detection

PCA-subspace method has been proposed for network-wide anomaly detection. Normal subspace contamination is still a great challenge for PCA although some methods are proposed to reduce the contamination. In this paper, we apply PCA-subspace method to six-month Origin-Destination (OD) flow data from the Abilene. The result shows that normal subspace contamination is mainly caused by anomalies from a few strongest OD flows, and seems unavoidable for subspace method. Further comparison of anomalies detected by subspace method and manually tagged anomalies from each OD flows, we find that anomalies detected by subspace method are mainly caused by anomalies from medium and a few large OD flows, and most anomalies of minor OD flows are buried in abnormal subspace and hard to be detected by PCA-subspace method. We analyze the reason for those anomalies undetected by subspace method and suggest to use normal subspace to detect anomalies caused by a few strongest OD flows, and to further divide abnormal subspace to detect more anomalies from minor OD flows. The goal of this paper is to address limitations neglected by prior works and further improve the subspace method on one hand, also call for novel detection methods for network-wide traffic on another hand.

IP traffic classification based on machine learning

With the rapid development of Internet, many network applications (e.g., P2P) use dynamic ports and encryption technology, which makes the traditional port and payload-based classification methods ineffective. Hence, it is important and necessary to find the more effective ones. Currently the machine learning (ML) techniques provide a promising alternative one for IP traffic classification. In this work, we use the ML-based classification method to identify the classes of the unknown flows using the payload-independent statistical features such as packet-length and arrival-interval. In order to improve the efficiency of the classification methods, the feature reduction techniques are further adopted to refine the selected features for attaining a best group of features. Finally we compare and evaluate the ML classification algorithms based on the BRASIL data source in terms of the three metrics such as overall accuracy, average precision and average recall. Our experiments show that the decision-tree algorithm is the best ML one for IP traffic classification and is able to construct the real-time classification system.

AMIR: Another Multipath Interdomain Routing

Multipath routing is an important and promising technique to increase the Internets reliability and to give users greater control over the service they receive. Currently the interdomain routing protocol limits each router to using a single route for a destination network, which does not satisfy the diverse requirements of end users. In this paper, in order to support the effective and efficient multipath routing, we propose a multipath interdomain routing system(AMIR), which not only provides more novel paths but also realizes a new AS-level routing scheme. In the control plane, the topology information is collected from neighboring ASes around the primary path, and based on this topology, multipath of the special node pairs is calculated by our multipath discovery algorithm. In the data plane, we use the interdomain source routing to forward the packets. Experiments with Internet topology and routing data demonstrate that AMIR is practical and feasible, and offers tremendous flexibility and diversity for path selection with reasonable overhead.

What's going on in Chinese IPv6 world

With IPv4 addresses quickly dwindling, the Internet is forcing an evolution of itself. During the long term transition from IPv4 to IPv6, whats going on in IPv6 world becomes unknown for network operators and researchers. In this paper, we propose a heuristic algorithm to identify p2p traffic accurately and implement traffic classification based on Netflow v9 exports to illustrate what applications Chinese IPv6 users are really running. Additionally, we present a detailed study of p2p traffic over IPv6 and advice ISPs to localize p2p traffic at the AS level for future IPv6 traffic management and network resources planning, leaving modeling traffic behavior and deeper classification of IPv6 traffic as our future work.

Multipath interdomain routing via deviation from primary path

Multi-path routing is a promising technique to increase the Internets reliability and to give users greater control over the service they receive. Currently the interdomain routing protocol limits each router to using a single route for same destination, which may not satisfy the diverse requirements of end users. In this paper, in order to support the effective and efficient multi-path service(MPS), we propose a multi-path inter-domain routing via deviation from primary path(MIR-DPP) which includes two main steps as following. Firstly, to improve the path diversity of node pairs, the special topology collected from neighboring ASes around the primary path, and the multipath set is calculated by our multipath discovery method. Secondly, to gain high-quality paths which are not compliant with routing policy, a negotiation mechanism is designed to address policy limits. Experiments with Internet topology and routing data demonstrate that MIR-DPP offers tremendous flexibility and diversity for path selection with reasonable overhead.

MCST: Anomaly detection using feature stability for packet-level traffic

In this paper, we present a statistical analysis of six traffic features based on entropy and distinct feature number at the packet level, and we find that, although these traffic features are unstable and show seasonal patterns like traffic volume for a long period, they are stable and consistent with Gaussian distribution in a short time period. However, this equilibrium property will be violated by some anomalies. Based on this observation, we propose a Multi-dimensional Clustering method for Short-time scale Traffic(MCST) to classify abnormal and normal traffic. We compare our new method to the well known wavelet technique. The detection result on synthetic anomaly traffic shows MCST can better detect the low-rate attacks than wavelet-based method, and detection result on real traffic demonstrates that MCST can detect more anomalies with low false alarm rate.

An efficient parallel TCAM scheme for the forwarding engine of the next-generation router

Ternary Content-Addressable Memory (TCAM) is a popular hardware device for fast IP address lookup. High link transmission speed of Internet backbone demands more powerful IP address lookup engine. Restricted by the memory access speed, the lookup engine for next-generation routers demands exploiting parallelism among multiple TCAM chips. However, most existing schemes improve lookup performance and reduce power consumption but ignore the update efficiency. In this paper, we propose a crossed address range division and shared caching scheme. We improve the update efficiency significantly by buddy update method while keep low power dissipation by decreasing the number of the triggered TCAMs access in each lookup operation. The lookup throughput is ultra high through adaptive load balance. Our simulation results show that the proposed scheme can achieve an average lookup speedup factor greater than 11 with 12 TCAM chips, on the cost of 10% more memory space and an additional cache chip.

Diagnosing Traffic Anomalies Using a Two-Phase Model

Network traffic anomalies are unusual changes in a network, so diagnosing anomalies is important for network management. Feature-based anomaly detection models (ab)normal network traffic behavior by analyzing packet header features. PCA-subspace method (Principal Component Analysis) has been verified as an efficient feature-based way in network-wide anomaly detection. Despite the powerful ability of PCA-subspace method for network-wide traffic detection, it cannot be effectively used for detection on a single link. In this paper, different from most works focusing on detection on flow-level traffic, based on observations of six traffic features for packet-level traffic, we propose a new approach B6-SVM to detect anomalies for packet-level traffic on a single link. The basic idea of B6-SVM is to diagnose anomalies in a multi-dimensional view of traffic features using Support Vector Machine (SVM). Through two-phase classification, B6-SVM can detect anomalies with high detection rate and low false alarm rate. The test results demonstrate the effectiveness and potential of our technique in diagnosing anomalies. Further, compared to previous feature-based anomaly detection approaches, B6-SVM provides a framework to automatically identify possible anomalous types. The framework of B6-SVM is generic and therefore, we expect the derived insights will be helpful for similar future research efforts.

FlowInfra: A fault-resilient scalable infrastructure for network-wide flow level measurement

The fine-grained flow level measurement is getting increasing demand in recent years. Though it fails to be a generic solution for its biased sampling, NetFlow is promising for its compatibility with major routers and its convenience to perform direct flow level measurement of both IPv4 and IPv6 traffic. Traditional flow level measurement systems based on NetFlow are mostly centralized and each of them independently performs traffic analysis of its local flow records without any coordination in a large-scale network, suffering from unbalancing workload and bad scalability. In this paper we present the design, implementation and evaluation of FlowInfra which is a fault-resilient scalable infrastructure for network-wide flow measurement of pure IPv6 flow records from NetFlow v9 exports. Through the assessment of its performance and flexible features, we show that FlowInfra achieved enhanced ability and robustness to perform network-wide flow level measurement and satisfied the goal for IPv6 network operation and management with better scalability.

Self-similar nature of the internet traffic information

Network traffic feature is generally described as self-similarity in a time series of volume counts (e.g. of bytes or packets) view. What is omitted from this view of traffic is the content of packets. In this paper, we find the self-similarity also exists in traffic information through long-time statistics of the internet traffic data, coupled with a discussion of the underlying mathematical and statistical properties of self-similarity and their relationship with actual network behavior. The goal of our paper is to open doors to: 1) further study properties of the packet information; 2) new traffic models and applications considering both volume and information.