Toshio Hirotsu

A PCA Analysis of Daily Unwanted Traffic

This paper investigates the macroscopic behavior of unwanted traffic (e.g., virus, worm, backscatter of (D)DoS or misconfiguration) passing through the Internet. The data set we used are unwanted packets measured at /18 darknet in Japan from Oct. 2006 to Apr. 2009 that included the recent Conficker outbreak. The traffic behavior is quantified by the entropy of ten packet features (e.g., 5-tuple). Then, we apply PCA (principal component analysis) to a ten dimensional entropy time series matrix to obtain a suitable representation of unwanted traffic. PCA is a well-known and studied method for finding out normal and anomalous behaviors in Internet backbone traffic, however, few studies applied it to darknet traffic. We first demonstrate the high variability nature of the entropy time series for ten packet features. Next, we show that the top four principal components are sufficiently enough to describe the original traffic behavior. In particular, the first component can be interpreted as the type of unwanted traffic (i. e., worm/virus or scanning), and the second one as the difference in communication patterns (e. g., one-to-many or many-to-one). Those two components account for 63.8\% of the original data set in terms of the total variance. On the other hand, the outliers in the higher components indicate the presence of specific anomalies although most of mapped data to the components have less variability. Furthermore, we show that the scatter plot of the first and second principal component scores provides us with a better view of the macroscopic unwanted traffic behavior.

Effect of alternative distributed task allocation strategy based on local observations in contract net protocol

This paper presents a distributed task allocation method whose strategies are alternatively selected based on the estimated workloads of the local agents. Recent Internet, sensor-network, and cloud computing applications are large-scale and fully-distributed, and thus, require sophisticated multi-agent system technologies to enable a large number of programs and computing resources to be effectively used. To elicit the capabilities of all the agents in a large-scale multi-agent system (LSMAS) in which thousands of agents work concurrently requires a new negotiation strategy for appropriately allocating tasks in a distributed manner. We start by focusing on the contract net protocol (CNP) in LSMAS and then examine the effects of the awardee selection strategies, that is, the task allocation strategies. We will show that probabilistic awardee selections improve the overall performance in specific situations. Next, the mixed strategy in which a number of awardee selections are alternatively used based on the analysis of the bid from the local agents is proposed. Finally, we show that the proposed strategy does not only avoid task concentrations but also reduces the wasted efforts, thus it can considerably improve the performance.

Predicting network throughput for grid applications on network virtualization areas

Grid applications are increasingly becoming dependent on network resources. Predicted network throughput is a useful parameter for network-aware scheduling for such applications. Although throughput prediction methods have been proposed, many of these methods are suffering from the fact that the probability distribution of traffic is unclear and the scale and bandwidth of networks are constantly changing. Furthermore, a virtual machine has been used as a platform for grid computing, and it can affect network measurement. A prediction method that uses pairs of differently sized connections has been proposed. This method, which we call connection pair, features a small probe transfer that predicts the throughput of a large data transfer. We propose a throughput prediction method based on the connection pair that uses v-support vector regression (SVR) and polynomial kernel to deal with prediction models represented as a non-linear and continuous monotonic function. The prediction accuracy of our method compared to that of a previous prediction method is higher. Moreover, the drop in the accuracy is also smaller than that of the previous method under an unstable network state. We clarify the prediction accuracy with other probe sizes for the connection pair. The accuracy is decreased by a small-sized probe, and there are no changes with a large-sized probe. These results show that our method is accurate, robust, and suitable for its purpose.

Dynamic and distributed routing control for virtualized local area networks

Advanced Layer-3 (L3) switches achieve high-speed IP packet forwarding by storing parts of the header information from transmitted packets into the flow cache in the switch fabric when relaying the IP packets between subnets. When IP traffic is overloaded on an L3 switch, the flow cache is easily exhausted, decreasing the IP packet forwarding performance. Virtual LAN (VLAN), a virtualization technology at the data-link layer, is widely used for the internal networks of many organizations because it allows network configurations to be changed easily and provides design flexibility. In a VLAN-based local area network with multiple L3 switches, the relaying point for each VLAN can be placed on any of the L3 switches. We developed a new network control scheme called distributed virtual routing, which dynamically controls the packet exchange points for each VLAN to suppress the consumption of flow caches. We describe the basic concept and then evaluate the reduction of the relaying flows through the simulation using the real network data.

Analysis of Anomalies on a Virtualized Network Testbed

To clarify useful parameters for avoiding unstable conditions in network experiments on a virtualized testbed, we used PlanetLab as the virtualized testbed and measured network throughput using a combination of probe and data transfers. Although PlanetLab has been widely used as a testbed for overlay networks, distributed systems, and network measurement, it is provided as a virtualized environment to users. A set of these environments on different nodes is called as ‘slice’, and multiple slices run simultaneously on each node. We found that network throughput was occasionally decreased even though the network condition was stable. The cause of the throughput decrease was an unintended large packet spacing. The unintended large packet spacing is an anomaly. Although the cause of the anomaly is known to be unstable CPU scheduling latency, no clear conditions for anomaly avoidance had previously been given. We investigated throughput measurement with resource monitoring to clarify anomaly avoidance conditions. When the CPUs at a node are shared by many slices, slices are frequently scheduled off the CPUs, and the anomaly occurs. If network throughput is decreased by the anomaly, the measurement results should be discarded.

Adaptive Routing Point Control in Virtualized Local Area Networks Using Particle Swarm Optimizations

This paper describes methods for controlling routing points of VLAN domains using binary particle swarm optimization (BPSO) and angle modulated particle swarm optimization (AMPSO). Virtual LAN (VLAN) is a technique for virtualizing data link layer (or L2) and can construct arbitrary logical networks on top of a physical network. However, VLAN often causes much redundant traffic due to inappropriate deployments of network-layer (L3) routing capabilities in VLAN networks. We propose two methods using BPSO and AMPSO, and show that they can adaptively select the routing points dynamically in accordance with the observed traffic patterns and thus reduce the redundant traffic. The convergence features are compared with those of the conventional method on the basis of a statistical method. Then we also show that the scalability of the algorithm using AMPOS is high and thus we can expect that it is applicable to practical large VLAN environments.

A Visualization Tool for Exploring Multi-scale Network Traffic Anomalies

Visualization is an intuitive and powerful way of understanding the evolution of huge amounts of network traffic in terms of characterizing network anomalies. We propose an interactive tool to display, explore, and understand network traffic focusing on anomalies. It displays traffic on different temporal and spatial (address and port) scales and lets users navigate network data by using a simple interface. Different graphical representations are used to highlight anomalies quickly, and textual packet information about corresponding plotted points are provided. The proposed tool provides good support for understanding traffic behavior and for evaluating the effectiveness of anomaly detection method. The tool directly reads dump files and uses no intermediate database in daily operations. This paper demonstrates several examples emphasizing specific patterns for various anomalies.

A Light-Weight Autonomous Power Saving Method for Wireless Sensor Networks

This paper proposes an autonomous energy saving method that works on wireless sensor networks. Wireless sensor nodes are equipment for gathering information about the users surrounding environment. An energy-efficient data gathering system is required because each node is battery powered. Our method autonomously reduces the sampling frequency for monitoring environments whose condition may vary rapidly or slowly. Its simple lightweight computation scheme also helps to reduce battery consumption. We conducted experiments using real sensor data and found that our method performed well in terms of both power consumption and quality of sampled data.

Performance Implications of Task Scheduling by Predicting Network Throughput on the Internet

A meta-scheduler is used to efficiently assign tasks to distributed resources. Additionally, the Internet is often used to share the data of tasks on the resources. Throughput prediction will play a crucial role in overcoming the instability of network resource on the Internet for the meta-scheduler. However, it is unclear that higher prediction accuracy should always guarantee better scheduling. In this paper, we focus on how predictors can affect the overall processing time of given tasks through meta-scheduler simulation. Real traces of throughput on the Internet are used to build CDF-based and SVR-based predictors, and these are adopted by the meta-scheduler. Through the simulation, the meta-scheduler using the predictor clearly reduces the processing time. Moreover, the meta-scheduler using the SVR-based, which performed better than CDF-based did in terms of throughput prediction, was observed to result in a reduction of up to 13.3% in the processing time compared to the meta-scheduler without any predictions. The expected value of performance improvement with the SVR-based predictor was calculated as 3.73%, while the value of CDF-based was 2.53%. On the other hand, the meta-scheduler using the predictor can seldom assign tasks to inappropriate sites due to only a few of inaccurate prediction results. As a result, the processing time is drastically increased in comparison with the meta-scheduler without any predictions.

Adaptive probabilistic task allocation in large-scale multi-agent systems and its evaluation

In this paper, we introduce the probabilistic awardee selection strategy, under which awardee is selected with a fixed probability, into the award phase of contract net protocol. We then point out that, by changing the probabilities in this strategy according the local workload, the overall performance can be considerably improved.