Karel Bartos

Adaptive Multiagent System for Network Traffic Monitoring

Individual anomaly-detection methods for monitoring computer network traffic have relatively high error rates. An agent-based trust-modeling system fuses anomaly data and progressively improves classification to achieve acceptable error rates.

Learning Detector of Malicious Network Traffic from Weak Labels

We address the problem of learning a detector of malicious behavior in network traffic. The malicious behavior is detected based on the analysis of network proxy logs that capture malware communication between client and server computers. The conceptual problem in using the standard supervised learning methods is the lack of sufficiently representative training set containing examples of malicious and legitimate communication. Annotation of individual proxy logs is an expensive process involving security experts and does not scale with constantly evolving malware. However, weak supervision can be achieved on the level of properly defined bags of proxy logs by leveraging internet domain black lists, security reports, and sandboxing analysis. We demonstrate that an accurate detector can be obtained from the collected security intelligence data by using a Multiple Instance Learning algorithm tailored to the Neyman-Pearson problem. We provide a thorough experimental evaluation on a large corpus of network communications collected from various company network environments.

IFS: Intelligent flow sampling for network security–an adaptive approach

In order to cope with an increasing volume of network traffic, flow sampling methods are deployed to reduce the volume of log data stored for monitoring, attack detection and forensic purposes. Sampling frequently changes the statistical properties of the data and can reduce the effectiveness of subsequent analysis or processing. We propose two concepts that mitigate the negative impact of sampling on the data. Late sampling is based on a simple idea that the features used by the analytic algorithms can be extracted before the sampling and attached to the surviving flows. The surviving flows thus carry the representation of the original statistical distribution in these attached features. The second concept we introduce is that of adaptive sampling. Adaptive sampling deliberatively skews the distribution of the surviving data to over-represent the rare flows or flows with rare feature values. This preserves the variability of the data and is critical for the analysis of malicious traffic, such as the detection of stealthy, hidden threats. Our approach has been extensively validated on standard NetFlow data, as well as on HTTP proxy logs that approximate the use-case of enriched IPFIX for the network forensics. Copyright

Self-Organized Mechanism for Distributed Setup of Multiple Heterogeneous Intrusion Detection Systems

We propose a distributed and self-organized framework for collaboration of multiple heterogeneous IDS sensors. The framework is based on a game-theoretical approach that optimizes behavior of each IDS sensor with respect to other sensors in highly dynamic environments. We formalize the proposed collaborative architecture as a game between defenders and attackers and transform the hard problem of heterogeneous collaboration into an easier problem of finding two functions that are used in the game-theoretical model to specialize the detection mechanisms on a specific type of malicious activity. The collaboration of such more specialized IDS nodes covers much wider range of attack classes, allowing the collaborating system to maximize the overall network security awareness. We have evaluated the proposed concept on real networks, where we have shown considerable improvements in the detection capabilities of intrusion detection devices thanks to the proposed collaboration model.

IFS: Intelligent flow sampling for network security-an adaptive approach

In order to cope with an increasing volume of network traffic, flow sampling methods are deployed to reduce the volume of log data stored for monitoring, attack detection and forensic purposes. Sampling frequently changes the statistical properties of the data and can reduce the effectiveness of subsequent analysis or processing. We propose two concepts that mitigate the negative impact of sampling on the data. Late sampling is based on a simple idea that the features used by the analytic algorithms can be extracted before the sampling and attached to the surviving flows. The surviving flows thus carry the representation of the original statistical distribution in these attached features. The second concept we introduce is that of adaptive sampling. Adaptive sampling deliberatively skews the distribution of the surviving data to over-represent the rare flows or flows with rare feature values. This preserves the variability of the data and is critical for the analysis of malicious traffic, such as the detection of stealthy, hidden threats. Our approach has been extensively validated on standard NetFlow data, as well as on HTTP proxy logs that approximate the use-case of enriched IPFIX for the network forensics. Copyright

Game Theoretical Model for Adaptive Intrusion Detection System

We present a self-adaptation mechanism for network intrusion detection system based on the use of game-theoretical formalism. The key innovation of our method is a secure runtime definition and solution of the game and real-time use of game solutions for immediate system reconfiguration. Our approach is suited for realistic environments where we typically lack any ground truth information regarding traffic legitimacy/maliciousness and where the significant portion of system inputs may be shaped by the attacker in order to render the system ineffective. Therefore, we rely on the concept of challenge insertion: we inject a small sample of simulated attacks into the unknown traffic and use the system response to these attacks to define the game structure and utility functions. This approach is also advantageous from the security perspective, as the manipulation of the adaptive process by the attacker is far more difficult.