Guangxing Zhang

Scalable high-performance parallel design for network intrusion detection systems on many-core processors

Network Intrusion Detection Systems (NIDSes) face significant challenges coming from the relentless network link speed growth and increasing complexity of threats. Both hardware accelerated and parallel software-based NIDS solutions, based on commodity multi-core and GPU processors, have been proposed to overcome these challenges. This work explores new parallel opportunities afforded by many-core processors for high performance, scalable and inexpensive NIDS. We exploit the huge many-core computational power by adopting a hybrid parallel architecture combining data and pipeline parallelism. We also design a hybrid load balancing scheme, using both ruleset and flow space partitioning. Furthermore, the proposed design leverages particular features of the processor to break the bottlenecks. We have integrated the open source NIDS Suricata into our proposed design and evaluated its performance with synthetic traffic. The prototype exhibits almost linear speedup and can handle up to 7.2 Gbps traffic with 100-bytes packets.

Survey on traffic of metro area network with measurement on-line

Network traffic measurements can provide essential data for network research and operation. While Internet traffic has been heavily studied for several years, there are new characteristics of traffic having not been understood well brought by new applications for example P2P. It is difficult to get these traffic metrics due to the difficulty to measurement traffic on line for high speed link and to identify new applications using dynamic ports. In this paper, we present a broad overview of Internet traffic of an operated OC-48 export link of a metro area network from a carrier with the method of measurement on-line. The traffic behaves a daily characteristic well and the traffic data of whole day from data link layer to application layer is presented. We find the characteristics of traffic have changed greatly from previous measurements. Also, we explain the reasons bringing out these changes. Our goal is to provide the first hand of traffic data that is helpful for people to understand the change of traffic with new applications.

Sequential and adaptive sampling for matrix completion in network monitoring systems

End-to-end network monitoring is essential to ensure transmission quality for Internet applications. However, in large-scale networks, full-mesh measurement of network performance between all transmission pairs is infeasible. As a newly emerging sparsity representation technique, matrix completion allows the recovery of a low-rank matrix using only a small number of random samples. Existing schemes often fix the number of samples assuming the rank of the matrix is known, while the data features thus the matrix rank vary over time. In this paper, we propose to exploit the matrix completion techniques to derive the end-to-end network performance among all node pairs by only measuring a small subset of end-to-end paths. To address the challenge of rank change in the practical system, we propose a sequential and information-based adaptive sampling scheme, along with a novel sampling stopping condition. Our scheme is based only on the data observed without relying on the reconstruction method or the knowledge on the sparsity of unknown data. We have performed extensive simulations based on real-world trace data, and the results demonstrate that our scheme can significantly reduce the measurement cost while ensuring high accuracy in obtaining the whole network performance data.

Accurate recovery of Internet traffic data: A tensor completion approach

The inference of traffic volume of the whole network from partial traffic measurements becomes increasingly critical for various network engineering tasks, such as traffic prediction, network optimization, and anomaly detection. Previous studies indicate that the matrix completion is a possible solution for this problem. However, as a two-dimension matrix cannot sufficiently capture the spatial-temporal features of traffic data, these approaches fail to work when the data missing ratio is high. To fully exploit hidden spatial-temporal structures of the traffic data, this paper models the traffic data as a 3-way traffic tensor and formulates the traffic data recovery problem as a low-rank tensor completion problem. However, the high computation complexity incurred by the conventional tensor completion algorithms prevents its practical application for the traffic data recovery. To reduce the computation cost, we propose a novel Sequential Tensor Completion algorithm (STC) which can efficiently exploit the tensor decomposition result for the previous traffic data to deduce the tensor decomposition for the current data. To the best of our knowledge, we are the first to apply the tensor to model Internet traffic data to well exploit their hidden structures and propose a sequential tensor completion algorithm to significantly speed up the traffic data recovery process. We have done extensive simulations with the real traffic trace as the input. The simulation results demonstrate that our algorithm can achieve significantly better performance compared with the literature tensor and matrix completion algorithms even when the data missing ratio is high.

Self-Similar Characteristic of Traffic in Current Metro Area Network

Complexity and diversity of Internet traffic are constantly growing. Networking researchers become aware of the need to constantly monitor and reevaluate their assumptions in order to ensure that the conceptual models correctly represent reality. Using the dataset collected by NetTurbo from three different bidirectional OC-48 links in metro area networks at the two biggest ISPs of China, this paper carefully investigates the self-similar characteristics of traffic from different aspects. In contrast to the previous results which have been widely accepted, this paper shows that for the aggregated traffic and the TCP and UDP traffic whether the self-similarity exists is uncertain. Further, break down by the application category, only the traditional and uncategorized traffic are self-similar while the others are not. However, on the view of the individual application of each category, it seems that traffic of every application exhibits self-similarity. To the best of our knowledge, this paper firstly provides the experimental evidence showing that aggregating different groups of self-similar traffic series could generate a traffic series which is either self-similar or non-self-similar.

Mnemonic Lossy Counting: An efficient and accurate heavy-hitters identification algorithm

Identifying heavy-hitter traffic flows efficiently and accurately is essential for Internet security, accounting and traffic engineering. However, finding all heavy-hitters might require large memory for storage of flows information that is incompatible with the usage of fast and small memory. Moreover, upcoming 100Gbps transmission rates make this recognition more challenging. How to improve the accuracy of heavy-hitters identification with limited memory space has become a critical issue. This paper presents a scalable algorithm named Mnemonic Lossy Counting (MLC) that improves the accuracy of heavy-hitters identification while having a reasonable time and space complexity. MLC algorithm holds potential candidate heavy-hitters in a historical information table. This table is used to obtain tighter error bounds on the estimated sizes of candidate heavy-hitters. We validate the MLC algorithm using real network traffic traces, and we compared its performance with two state-of-the-art algorithms, namely Lossy Counting (LC) and Probabilistic Lossy Counting (PLC). The results reveal that: 1) with same set of parameters and memory usage, MLC achieves between 31.5% and 6.67% fewer false positives than LC and PLC. 2) MLC and LC have a zero false negative ratio, whereas 38% of the cases PLC has a non-zero false negatives and PLC can miss up to 4.4% of heavy-hitters. 3) MLC has a slightly lower memory cost than LC during the first few windows and its memory usage decreases with time, when PLC memory usage declines sharply. 4) MLC has similar runtime than LC, and smaller time than PLC.

Rogue access point detection using segmental TCP jitter

Rogue Access Points (RAPs) pose serious security threats to local networks. An analytic model of prior probability distribution of Segmental TCP Jitter (STJ) is deduced from the mechanism of IEEE 802.11 MAC Distributed Coordinated Function (DCF) and used to differentiate the types of wire and WLAN connections which is the crucial step for RAPs detecting. STJ as the detecting metric can reflect more the characteristic of 802.11 MAC than ACK-Pair since it can eliminate the delay caused by packet transmission. The experiment on an operated network shows the average detection ratio of the algorithm with STJ is more than 92.8% and the average detection time is less than 1s with improvement of 20% and 60% over the detecting approach of ACK-Pair respectively. Farther more no WLAN training trace is needed in the detecting algorithm.

Design and Implementation of a Network Behavior Analysis-Oriented IP Network Measurement System

Analyzing the characteristic of network behavior provides scientific basis for designing, building, and managing the next generation Internet, and is especially important for monitoring network behavior. This paper establishes a system of metrics that evaluates the behavior of IP networks with respect to the need of analyzing network behavior, introduces the design and implementation of network monitoring system that focuses on the analysis of the characteristics of network behavior, analyzes crucial problems on system design, builds an experiment environment and runs tests on it. The results show that our system satisfies all requirements imposed by real time monitoring network behavior, therefore is able to help the decision making in operating and managing networks.

An adaptive method for identifying heavy hitters combining sampling and data streaming counting

Identifying heavy hitters is essential for network monitoring, management, charging and etc. Existing methods in the literature have some limitations. How to reduce the memory consumption effectively without compromising identification accuracy is still challenging. In this paper, an adaptive method combining sampling and data streaming counting is proposed, called FSPLC(feedback sampling probabilistic lossy counting). Based on the history information in the flow counter table, FSPLC can adjust the sampling frequency dynamically, and also adapt to the real-time traffic changes. Comparison with state-of-the-art algorithms based on real Internet traces suggests that FSPLC is remarkably efficient and accurate. Experiment results show that FSPLC has 1) 60% lower memory consumption, 2) 15% smaller false-positive ratio.

Throughput Guaranteed Handoff for SDN-Based WLAN in Distinctive Signal Coverage

In SDN-based WLAN, controller needs to collect the state info of Mobile Nodes (MNs) like received signal strength indicator (RSSI) for handoff. In such scenarios, high sampling rate facilitates handoff, but it also easily leads to system overhead thus limits access scale of MN. Besides, dynamical adjustment of transmit power of access point (AP) leads to the distinctive signal coverage. Existing handoff algorithms that directly use the uplink RSSI as handoff condition would result in significant throughput decay of MN. Also in indoor deployment RSSI may varies much, large variation of RSSI result in unstable handoff. To address the issues, we design a variable sampling rate mechanism, then filter sampling RSSI and propose a handoff algorithm for distinctive signal coverage scenarios. Our sampling mechanism uses a finite state machine (FSM) to adjust the sampling rate by MN on all nearby APs. Our handoff algorithm uses Kalman filter to achieve stable and trend-reflecting uplink RSSI estimation, then estimate downlink signal noise ratio (SNR) difference between potential and current AP. We implement our algorithm and deployed a test-bed for extensive experiments. Results show our sampling mechanism could achieve sampling quantity decrease by 60%; compared to mean filter based approach, our handoff algorithm improves throughput by 10-50% in distinctive signal coverage scenarios. Besides, handoff frequency decreased by about 60%, indicating a more stable handoff decision.