Todd S. Sproull

Deep packet inspection using parallel Bloom filters

Recent advances in network packet processing focus on payload inspection for applications that include content-based billing, layer-7 switching and Internet security. Most of the applications in this family need to search for predefined signatures in the packet payload. Hence an important building block of these processors is string matching infrastructure. Since conventional software-based algorithms for string matching have not kept pace with high network speeds, specialized high-speed, hardware-based solutions are needed. We describe a technique based on Bloom filters for detecting predefined signatures (a string of bytes) in the packet payload. A Bloom filter is a data structure for representing a set of strings in order to support membership queries. We use hardware Bloom filters to isolate all packets that potentially contain predefined signatures. Another independent process eliminates false positives produced by Bloom filters. We outline our approach for string matching at line speeds and present a performance analysis. Finally, we report the results for a prototype implementation of this system on the FPX platform. Our analysis shows that with the state-of-the-art FPGAs, a set of 10,000 strings can be scanned in the network data at the line speed of OC-48 (2.4 Gbps).

Deep packet inspection using parallel bloom filters

There is a class of packet processing applications that inspect packets deeper than the protocol headers to analyze content. For instance, network security applications must drop packets containing certain malicious Internet worms or computer viruses carried in a packet payload. Content forwarding applications look at the hypertext transport protocol headers and distribute the requests among the servers for load balancing. Packet inspection applications, when deployed at router ports, must operate at wire speeds. With networking speeds doubling every year, it is becoming increasingly difficult for software-based packet monitors to keep up with the line rates. We describe a hardware-based technique using Bloom filters, which can detect strings in streaming data without degrading network throughput. A Bloom filter is a data structure that stores a set of signatures compactly by computing multiple hash functions on each member of the set. This technique queries a database of strings to check for the membership of a particular string. The answer to this query can be false positive but never a false negative. An important property of this data structure is that the computation time involved in performing the query is independent of the number of strings in the database provided the memory used by the data structure scales linearly with the number of strings stored in it. Furthermore, the amount of storage required by the Bloom filter for each string is independent of its length.

Scalable IP lookup for programmable routers

Continuing growth in optical link speeds places increasing demands on the performance of Internet routers, while deployment of embedded and distributed network services imposes new demands for flexibility and programmability. IP address lookup has become a significant performance bottleneck for the highest performance routers. Amid the vast array of academic and commercial solutions to the problem, few achieve a favorable balance of performance, efficiency, and cost. New commercial products utilize content addressable memory (CAM) devices to achieve high lookup speeds at an exorbitantly high hardware cost with limited flexibility. In contrast, this paper describes an efficient, scalable lookup engine design, able to achieve high performance with the use of a small portion of a reconfigurable logic device and a commodity random access memory (RAM) device. The Fast Internet Protocol Lookup (FIPL) engine is an implementation of Eatherton and Dittias previously unpublished Tree Bitmap algorithm (1998) targeted to an open-platform research router. FIPL can be scaled to achieve guaranteed worst-case performance of over 9 million lookups per second with a single SRAM operating at the fairly modest clock speed of 100 MHz. Experimental evaluation of FIPL throughput, latency, and update performance is provided using a sample routing table from Mae West.

Scalable IP lookup for Internet routers

Internet protocol (IP) address lookup is a central processing function of Internet routers. While a wide range of solutions to this problem have been devised, very few simultaneously achieve high lookup rates, good update performance, high memory efficiency, and low hardware cost. High performance solutions using content addressable memory devices are a popular but high-cost solution, particularly when applied to large databases. We present an efficient hardware implementation of a previously unpublished IP address lookup architecture, invented by Eatherton and Dittia (see M.S. thesis, Washington Univ., St. Louis, MO, 1998). Our experimental implementation uses a single commodity synchronous random access memory chip and less than 10% of the logic resources of a commercial configurable logic device, operating at 100 MHz. With these quite modest resources, it can perform over 9 million lookups/s, while simultaneously processing thousands of updates/s, on databases with over 100000 entries. The lookup structure requires 6.3 bytes per address prefix: less than half that required by other methods. The architecture allows performance to be scaled up by using parallel fast IP lookup (FIPL) engines, which interleave accesses to a common memory interface. This architecture allows performance to scale up directly with available memory bandwidth. We describe the tree bitmap algorithm, our implementation of it in a dynamically extensible gigabit router being developed at Washington University in Saint Louis, and the results of performance experiments designed to assess its performance under realistic operating conditions.

Snort offloader: a reconfigurable hardware NIDS filter

Software-based network intrusion detection systems (NIDS) often fail to keep up with high-speed network links. In this paper an FPGA-based pre-filter is presented that reduces the amount of traffic sent to a software-based NIDS for inspection. Simulations using real network traces and the Snort rule set show that a pre-filter can reduce up to 90% of network traffic that would have otherwise been processed by Snort software. The projected performance enables a computer to perform real-time intrusion detection of malicious content passing over a 10 Gbps network using FPGA hardware that operates with 10 Gbps of throughput and software that needs only to operate with 1 Gbps of throughput.

Control and configuration software for a reconfigurable networking hardware platform

A suite of tools called NCHARGE (Networked Configurable Hardware Administrator for Reconfiguration and Governing via End-systems) has been developed to simplify the co-design of hardware and software components that process packets within a network of Field Programmable Gate Arrays (FPGAs). A key feature of NCHARGE is that it provides a high-performance packet interface to hardware and standard Application Programming Interface (API) between software and reprogrammable hardware modules. Using this API, multiple software processes can communicate to one or more hardware modules using standard TCP/IP sockets. NCHARGE also provides a Web-Based User Interface to simplify the configuration and control of an entire network switch that contains several software and hardware modules.

Mutable codesign for embedded protocol processing

This paper addresses exploitation of the capabilities of platform FPGAs to implement embedded networking for systems on chip. In particular, a methodology for exploring trade-offs between the placement of protocol handling functions in programmable logic and on an embedded processor is demonstrated. This is facilitated by two new design tool capabilities: first, being able to describe programmable logic based functions in a more software-like manner; and second, being able automatically to generate efficient interfaces between a programmable logic fabric and an embedded processor. The methodology is illustrated by an example of a simple web server, targeted at Xilinx Virtex-II Pro or Virtex-4 FX platform FPGAs. Trade-offs both of complete protocol placement and of within-protocol placement are systematically investigated in terms of resources used and packet handling latency. This provides an excellent range of service times, corresponding to differing logic fabric and memory resource requirements. The work points the way to highly fluid allocation of functions to implementations, beyond conventional static codesign.

Sensor fusion and correlation

The sensor fusion architecture is a General-purpose Aggregation Processor (GAP) designed to bridge the gap between low-level sensor data and the high-level knowledge needed by the backbone. By examining both the incoming data from attached sensor nodes and the interests of users on the network, the model reduces overall transmission costs by keeping local event information at the source, only reporting the higher-level alerts and knowledge to interested parties. The sensor fusion architecture possesses three distinct advantages worth noting.

Extensible Network Configuration and Communication Framework

The effort to manage network security systems has increased in complexity over the past years. Network security for a company, university, or government agency can no longer be provided using a single Internet firewall or Intrusion Prevention System (IPS). Today, network administrators must deploy multiple intrusion detection and prevention nodes, traffic shapers, and firewalls in order to effectively protect their network. As the number of devices increases, maintaining a secure environment becomes difficult. This paper presents an infrastructure for control, configuration, and communication between heterogeneous network devices. The approach presented uses a Publish/Subscribe model built on top of a peer-to-peer overlay network in order to distribute information between network intrusion detection and prevention devices.

Management and Service Discovery in Satellite and Avionic Networks

Command and control services manage network-attached assets deployed in distributed systems that can be separated by thousands of miles. Networks that rely on satellite communications to transit all data to a centralized control center are troubled by high latency due to long propagation delays to satellites and limited data transit over bandwidth constrained links. Low latency communications can be achieved by using a combination of distributed airborne and space-based systems. This research investigates how deployment of a peer-to-peer (P2P) overlay network in a region of conflict can reduce the latency for real time control and communication. This overlay network utilizes a hybrid of both satellite and aircraft links to provide services that best satisfy the immediate needs of ground units. Experiments have been performed with an emulation testbed using 147 compute nodes in the Emulab testbed to study the latency and throughput of the overlay network. The overlay network is developed using a peer-to-peer application programmers interface (API) called JXTA. Nodes simulate resources requesting and offering several types of video and data services.