Shashank Shanbhag

Accurate anomaly detection through parallelism

In this article we discuss the design and implementation of a real-time parallel anomaly detection system. The key idea is to use multiple existing anomaly detection algorithms in parallel on thousands of network traffic subclasses, which not only enables us to detect hidden anomalies but also to increase the accuracy of the system. The main challenge then is the management and aggregation of the vast amount of data generated. We propose a novel aggregation process that uses the internal continuous anomaly metrics used by the algorithms to output a single system-wide anomaly metric. The evaluation on real-world attack traces shows a lower false positive rate and false negative rate than any individual anomaly detection algorithm.

Automated composition of data-path functionality in the future internet

Modern networks not only forward traffic, but also perform a variety of processing operations on packets (e.g., content inspection, transcoding, QoS scheduling). Such data-path functionality needs to be composed suitably to ensure correct operations and adherence to policies put in place by different entities in the Internet. Currently, there exists no explicit support for describing the semantics of packet processing services, for composing these services, and for ensuring that policies are considered. In our work, we propose a novel system for representing data-path functionality and policies in such a way that per-connection configurations can be composed automatically. We present the theoretical foundations of this approach as well as a prototype implementation based on our network service architecture. Our results show that this approach is an effective solution toward scalable and autonomic handling of data-path functionality in the future Internet. Index Terms next-generation Internet, network service, packet processing semantics, automated planning, data-path policies.

Automated Service Composition in Next-Generation Networks

Dynamic composition of protocol features allows applications to establish connections with custom communication characteristics. Automatically computing possible compositions and checking given compositions requires a common framework for expressing application needs, service features, and system characteristics. In this paper, we present such a framework that is based on situation calculus. We show that the automated composition problem can be reduced to an AI Planning problem. We further illustrate the effectiveness of this approach with several examples.

Virtual network mapping with traffic matrices

Network virtualization is a core technology in next-generation networks to overcome the ossification problem that is observed in the current Internet. The key idea of network virtualization is to split physical network resources into multiple logical networks, each supporting different network services and functionalities. One of the key challenges for virtual network infrastructure providers is to efficiently allocate network resources based on virtual network requests, which is referred to as the virtual network mapping problem. While several algorithms have been proposed previously to solve this mapping problem, their effectiveness is limited since virtual requests specify the internal topology of the virtual network. In this paper, we argue that such internal topologies lead to unnecessary constraints and less efficient solutions. Instead, we propose an alternate formulation of the problem that represents virtual network requests as traffic matrices. We provide a solutions to solving this traffic-matrix-based mapping problem using a mixed integer programming formulation. Our simulation results show that our approach can map significantly more virtual network requests on a physical network infrastructure than previous mapping algorithms and thus improves the efficient use of networking resources in virtual networks.

VHub: Single-stage virtual network mapping through hub location

Abstract Network virtualization allows multiple networks with different protocol stacks to share the same physical infrastructure. A key problem for virtual network providers is the need to efficiently allocate their customers’ virtual network requests to the underlying network infrastructure. This problem is known to be computationally intractable and heuristic solutions continue to be developed. Most existing heuristics use a two-stage approach in which virtual nodes are first placed on physical nodes and virtual links are subsequently mapped. In this paper, we present a novel approach to virtual network mapping that simultaneously maps virtual nodes and links onto the network infrastructure. Our VHub technique formulates the problem of mapping a virtual network request as a mixed integer program that is based on the p-hub median problem. Results from extensive simulations with synthetic and real virtual network requests show that our solution outperforms existing heuristics, including subgraph isomorphism backtracking search. Our approach requires fewer physical resources to accommodate virtual networks and is able to balance load more evenly across the network infrastructure.

Fair multithreading on packet processors for scalable network virtualization

Network virtualization requires careful control of networking resources, including link bandwidth, router memory, and packet processing time. Isolation and fair sharing of processing resources in current high-performance packet processors occur at the granularity of entire processor cores. Scaling of network virtualization to larger numbers of parallel slices requires a more fine-grained processor sharing mechanism. Our work presents a novel approach, called Fair Multithreading (FMT), that allows hardware threads to share a processor core while ensuring isolation and weighted fair access. We present an analysis of the FMT algorithm and a prototype implementation on a NetFPGA system. Our evaluation results indicate that FMT can be implemented at speeds that are necessary to make scheduling decisions at the instruction level. We show the impact of having such fine-grained processor schedulers in substrate nodes by comparing the resource utilization of virtual network slices in our system to traditional whole-core allocations. Our simulation results show the FMT-based substrate networks can be utilized more efficiently and more virtual network requests can be accommodated. These results indicate the significant improvement in system scalability that can be gained from our fine-grained processor scheduling system.

Massively Parallel Anomaly Detection in Online Network Measurement

Detecting anomalies during the operation of a network is an important aspect of network management and security. Recent development of high-performance embedded processing systems allow traffic monitoring and anomaly detection in real-time. In this paper, we show how such processing capabilities can be used to run several different anomaly detection algorithms in parallel on thousands of different traffic subclasses. The main challenge in this context is to manage and aggregate the vast amount of data generated by these processes. We propose (1) a novel aggregation process that uses continuous anomaly information (rather than binary outputs) from existing algorithms and (2) an anomaly tree representation to illustrate the state of all traffic subclasses. Aggregated anomaly detection results show a lower false positive and false negative rate than any single anomaly detection algorithm.

Evaluation of an Online Parallel Anomaly Detection System

The rapid and accurate detection of anomalies in network traffic has always been a challenging task, and is absolutely critical to the efficient operation of the network. The availability of numerous different detection algorithms makes it difficult to choose a suitable configuration. An algorithm may have a high detection rate for high rate attacks, but might behave unfavorably when faced with attacks with gradually increasing rates. This paper proposes an online parallel anomaly detection system that implements multiple anomaly detection algorithms in parallel to detect anomalies in real-time. The main idea is to aggregate the detection data from multiple algorithms to come up with a single anomaly metric. We evaluate this system with realistic attacks on the DETER testbed. Our results show improved true positive and false negative rates for both high intensity and slow-rise ramped floods. Furthermore, the system is able to detect attacks separated by as little as 15 seconds with a high true positive rate.

Enforcement of Data-Plane Policies in Next-Generation Networks

Modern networks not only forward traffic, but also perform a variety of processing operations on packets (e.g., content inspection, transcoding, QoS scheduling). Such data plane operations cannot be easily coordinated in the current Internet architectures since there is no explicit policy support for packet processing services. As more diverse systems and protocols are deployed in the next-generation Internet, this problem becomes increasingly challenging. In our work, we propose a novel policy enforcement system for data-path functions in the next-generation Internet. Using a formalism to represent policies and automated planning tools, connection request can be adapted to meet the policy requirement of the domains they traverse. We present the theoretical foundations of this approach as well as a prototype implementation based on our network service architecture. Our results show that this approach is an effective solution to enforcing policies relating to the date plane of networks.

Interfacing to a virtualized network infrastructure through network service abstractions

Virtualization is the base for a diversified next-generation network architecture design. Much work has been done on virtualization infrastructure, but it is still unclear how to easily instantiate a network slice that meets a high-level description of network functionality. Our work addresses this problem that occurs at the interface between network service providers and virtualized infrastructure providers. The proposed Service-based Network Virtualization System (SNVS), which is based on our previously developed network service architecture, provides a comprehensive and extensible interface to ease the development of new network services, protocols, and applications in virtualized networks.