Martin Kappes

Hierarchical events for efficient distributed network analysis and surveillance

Computer networks are elemental for todays Information Technology (IT) infrastructure. Maintaining operational computer networks is an important and challenging task. For this task the information gathered with means of network analysis and surveillance is the very basis. Detailed network analysis and surveillance is one of the key factors for assuring operational computer networks. However, network analysis and surveillance also poses many challenges like huge performance requirements or distributed application. In this paper we evaluate how event-based data processing and Complex Event Processing (CEP) can be leveraged for addressing these issues by enabling scalable event hierarchies.

Monitoring traffic in computer networks with dynamic distributed remote packet capturing

We present an approach for flexible distributed remote packet capturing with additional self-adaptivity and cooperation capabilities. Such techniques are needed for gaining comprehensive insight into large computer networks. With our system it is possible to operate multiple distributed remote packet capturing sensors from arbitrary locations. Advanced features like self-adaptivity or the cooperative use of sensors help to increase the performance. Empirical results obtained with a prototype indicate that our approach is efficient and allows capturing of traffic with speeds in the order of magnitude of Gigabit Ethernet. Furthermore, our approach integrates with and enables the use of existing packet processing applications.

Bridging the Gap between Low-Level Network Traffic Data Acquisition and Higher-Level Frameworks

The combination of low-level network data acquisition and higher-level frameworks results in more powerful and efficient network analysis and surveillance systems and opens up new possibilities for leveraging low-level network traffic data. There is, however, a gap between low-level network traffic data acquisition techniques and higher-level frameworks. This gap complicates the integration of low-level tools into higher-level frameworks. In this paper, we exemplarily research the feasibility of bridging this gap. As example, we use the integration of low-level packet capturing with Java-based higher-level implementations. To assess the practicability, we created a prototype implementation and performed extensive performance measurements. The results show that our Java-based prototype is capable of capturing and processing network traffic in near real-time at Gigabit Ethernet speed on consumer-class commodity hardware.

Improving network traffic acquisition and processing with the Java Virtual Machine

While network traffic acquisition and processing is typically done with languages like C that allow low-level hardware access and optimizations, languages like Java and their ecosystems aim at easing complex tasks. With a combination of both, strengths can be combined such that more powerful and versatile network traffic processing systems can be engineered. However, while approaches using languages like C are evolved and optimized, network traffic acquisition and processing with the Java Virtual Machine (JVM) is not equivalently optimized and benefits from employing JVM-based languages are not fully exploited yet. We present methods for increasing the network traffic processing performance with the JVM and examples for leveraging dynamic capabilities via a domain specific language and self-adaptivity. We measured improvements by factors of up to 5.9 compared to the old approach and capture rates up to 4.46 million packets per second and could show that the prototype is capable to self-adapt based on performance constraints.

Header Field Based Partitioning of Network Traffic for Distributed Packet Capturing and Processing

Maintaining correctly operating computer networks is paramount for assuring properly operating information technology infrastructures. Thereby, the acquisition of network traffic data is one of the first steps. The acquisition of network traffic, however, can be very challenging, e.g., with respect to performance and resource requirements. In this paper, we analyze the possibility of using packet header data for efficiently partitioning live network traffic data into subsets with the aim on enabling distributed packet capturing and processing. The goal is to employ multiple sensors in a coordinated fashion such that the overall task is distributed among the participating sensors. Our results show that efficiently partitioning live network traffic based on packet header data is possible. Furthermore, we implemented a prototype of a distributed packet capturing system that achieves significantly higher capture rates than a single, uncoordinated sensor.

Optimization of monitoring in dynamic communication networks using a hybrid evolutionary algorithm

In this paper, we propose a hybrid evolutionary algorithm (EA) for the optimization of efficient monitoring in dynamic communication networks. The first step towards improving communication infrastructures is gathering information about the current situation. One part of collecting this information is to implement an adequate monitoring in the network, i.e., the optimal positions and amount of monitoring devices, in order to analyze communication flows. Solving the general monitor selection problem using evolutionary computation has already been done in the past. Our approach focuses on the efficient optimization of monitors having a dynamic search landscape, i.e., having recurring substantial changes of the underlying network model in order to simulate bulks of entering or leaving nodes and edges. Here, we compare the steady optimization versions of a common genetic algorithm (GA), the proposed hybrid EA, and a local search based EA, in conjunction with a total restart version of the hybrid EA. Empirical results are obtained using multiple well-known real-world problem instances. We show that we can achieve reliably fast high quality results using the proposed hybrid EA.

Leveraging diversity in evolutionary algorithms using a population injection method

In this paper, we present a new, computationally inexpensive method for preventing premature convergence in multimodal evolutionary algorithms by population injection. Our method avoids the premature convergence of the population around one or multiple local optima by maintaining an adequate amount of genetic diversity. The technique does not require any setup or maintenance effort during runtime as is the case for other proposed techniques addressing the same issue (e.g., island, cellular, or diffusion model EAs as population models or specific operators for increasing genetic diversity in mutation and recombination). We present experimental results comparing a (μ, λ) EA using our method, which has been named population injection evolutionary algorithm (PI-EA), against cellular EA and classical (μ,λ) EA for some standard benchmark functions. In the results it can be observed that applying population injection improves the results produced by (μ, λ) EAs for all benchmarks under consideration, in one case even up to 59%.

Local parallelization of pleasingly parallel stream processing on multiple CPU cores

Data stream processing addresses the need for high-throughput near real-time data processing, which can be considered as one part of Big Data or Fast Data. In this paper, we study the local parallelization of stream processing on a single multi-core Central Processing Unit (CPU) computer system, which, in our opinion, was not sufficiently addressed yet. In distributed systems, optimizing the local throughput can help to improve the overall system. In less resource demanding scenarios, it may be beneficial to use more lightweight local parallelization instead of more complex distributed approaches. We present our work-in-progress on locally parallelizing stream processing on multiple CPU cores and on ways for further improving the local data processing. In order to study the fundamental mechanisms and effects, we focused on pleasingly parallel workloads. While pleasingly parallel tasks, by definition, can be easily parallelized, our results show that stream processing adds important aspects and that the outcomes strongly vary depending on use case and parallelization approach. Furthermore, we present early stages of a stream transformation Domain Specific Language and of a self-adaptive mechanism for easing and optimizing the processing. We published our implementations as Open Source Software.

Maintaining Genetic Diversity in Multimodal Evolutionary Algorithms using Population Injection

In this paper, we present a computationally inexpensive method for maintaining genetic diversity in evolutionary algorithms using population injection. As opposed to other methods, e.g., cellular EAs, population injection does not require any maintenance or setup effort. Here, we present first experimental results comparing a (μ, λ) EA with and without population injection and a cellular EA using the h1 benchmark. As can be observed in the results, population injection is worth to be considered for problems which suffer from premature convergence.

Using evolutionary dynamic optimization for monitor selection in highly dynamic communication infrastructures

In this paper, we address the problem of applying evolutionary dynamic optimization of network monitoring to highly dynamic communication network infrastructures. One major challenge of modern communication networks is the increasing volatility due to, e.g., changing availability of nodes and links, load of paths, or attacks. While optimization of those dynamic networks has been an important application area since decades, new developments in the area of network function virtualization and software defined network facilitate a completely new level of automated dynamic network optimization. Especially in mobile networks, changes can be observed to appear swiftly. Thus, using population-based heuristics becomes challenging as reevaluation of all candidate solutions may become time-wise impossible and operations need to rely on possibly obsolete fitness values. Here, an established method has been applied to solve the dynamic monitor selection problem on multiple real-world problem instances using a different simulated level of change. Statistically significant results of the proposed method have been compared to the performance of a best-of-multiple selection local search (EMS LS) heuristic. As the results show, optimization reaches results of high quality even under difficult circumstances.