Genevieve Bartlett

Census and survey of the visible internet

Prior measurement studies of the Internet have explored traffic and topology, but have largely ignored edge hosts. While the number of Internet hosts is very large, and many are hidden behind firewalls or in private address space, there is much to be learned from examining the population of visible hosts, those with public unicast addresses that respond to messages. In this paper we introduce two new approaches to explore the visible Internet. Applying statistical population sampling, we use censuses to walk the entire Internet address space, and surveys to probe frequently a fraction of that space. We then use these tools to evaluate address usage, where we find that only 3.6% of allocated addresses are actually occupied by visible hosts, and that occupancy is unevenly distributed, with a quarter of responsive /24 address blocks (subnets) less than 5% full, and only 9% of blocks more than half full. We show about 34 million addresses are very stable and visible to our probes (about 16% of responsive addresses), and we project from this up to 60 million stable Internet-accessible computers. The remainder of allocated addresses are used intermittently, with a median occupancy of 81 minutes. Finally, we show that many firewalls are visible, measuring significant diversity in the distribution of firewalled block size. To our knowledge, we are the first to take a census of edge hosts in the visible Internet since 1982, to evaluate the accuracy of active probing for address census and survey, and to quantify these aspects of the Internet.

Understanding passive and active service discovery

Increasingly, network operators do not directly operate computers on their network, yet are responsible for assessing network vulnerabilities to ensure compliance with policies about information disclosure, and tracking services that affect provisioning. Thus, with decentralized network management, service discovery becomes an important part of maintaining and protecting computer networks. We explore two approaches to service discovery: active probing and passive monitoring. Active probing finds all services currently on the network, except services temporarily unavailable or hidden by firewalls; however, it is often too invasive, especially if used across administrative boundaries. Passive monitoring can find transient services, but misses services that are idle. We compare the accuracy of passive and active approaches to service discovery and show that they are complimentary, highlighting the need for multiple active scans coupled with long-duration passive monitoring. We find passive monitoring is well suited for quickly finding popular services, finding servers responsible for 99% of incoming connections within minutes. Active scanning is better suited to rapidly finding all servers, which is important for vulnerability detection - one scan finds 98% of services in two hours, missing only a handful. External scans are an unexpected ally to passive monitoring, speeding service discovery by the equivalent of 9-15 days of additional observation. Finally, we show how the use of static or dynamic addresses changes the effectiveness of service discovery, both due to address reuse and VPN effects.

Inherent Behaviors for On-line Detection of Peer-to-Peer File Sharing

Blind techniques to detect network applications-approaches that do not consider packet contents-are increasingly desirable because they have fewer legal and privacy concerns, and they can be robust to application changes and intentional cloaking. In this paper we identify several behaviors that are inherent to peer-to-peer (P2P) traffic and demonstrate that they can detect both BitTorrent and Gnutella hosts using only packet header and timing information. We identify three basic behaviors: failed connections, the ratio of incoming and outgoing connections, and the use of unprivileged ports. We quantify the effectiveness of our approach using two day-long traces, achieve up to an 83% true positive rate with only a 2% false positive rate. Our system is suitable for on-line use, with 75% of new P2P peers detected in less than 10 minutes of trace data.

Experiences with a continuous network tracing infrastructure

One of the most pressing problems in network research is the lack of long-term trace data from ISPs. The Internet carries an enormous volume and variety of data; mining this data can provide valuable insight into the design and development of new protocols and applications. Although capture cards for high-speed links exist today, actually making the network traffic available for analysis involves more than just getting the packets off the wire, but also handling large and variable traffic loads, sanitizing and anonymizing the data, and coordinating access by multiple users. In this paper we discuss the requirements, challenges, and design of an effective traffic monitoring infrastructure for network research. We describe our experience in deploying and maintaining a multi-user system for continuous trace collection at a large regional ISP@. We evaluate the performance of our system and show that it can support sustained collection and processing rates of over 160--300Mbits/s.

Low-rate, flow-level periodicity detection

As desktops and servers become more complicated, they employ an increasing amount of automatic, non-user initiated communication. Such communication can be good (OS updates, RSS feed readers, and mail polling), bad (keyloggers, spyware, and botnet command-and-control), or ugly (adware or unauthorized peer-to-peer applications). Communication in these applications is often regular, but with very long periods, ranging from minutes to hours. This infrequent communication and the complexity of todays systems makes these applications difficult for users to detect and diagnose. In this paper we present a new approach to identify low-rate periodic network traffic and changes in such regular communication. We employ signal-processing techniques, using discrete wavelets implemented as a fully decomposed, iterated filter bank. This approach not only detects low-rate periodicities, but also identifies approximate times when traffic changed. We implement a self-surveillance application that externally identifies changes to a users machine, such as interruption of periodic software updates, or an installation of a keylogger.

Critter: Content-Rich Traffic Trace Repository

Access to current application and network data is vital to cybersecurity and networking research. Intrusion detection, steganography, traffic camouflaging, traffic classification and modeling all benefit from real-world data. Such data provides training, testing, and evaluation as well as furthers efforts to reach ground truth. Currently available network data--especially data with application-level information--is often outdated and is either private or customized to specific, narrow research needs. The biggest hurdle to obtaining such content-rich data is addressing the huge privacy risks associated with sharing such complex and open-ended data. In this paper we present a data sharing system called Critter-at-Home which addresses these challenges. Critter connects end-users willing to share data with researchers and strikes a balance between privacy risks for a data contributor and utility for a researcher.

Expressing Different Traffic Models Using the LegoTG Framework

In this paper we demonstrate the ease of generating and modifying background traffic in testbed experiments through the traffic generation framework we developed, called LegoTG. LegoTG is a modular framework for composing custom traffic generation. It makes it easy to combine different traffic generators and traffic modulators (e.g., Delay models), and to port the same background traffic to different topologies. In addition to the framework, we have developed several traffic generation/modulation tools that can be used in LegoTG to generate realistic and highly controllable network and transport-level traffic. We build our demonstration around a series of simple experiments which reinforce how much background traffic matters in experiments and how different traffic models can drastically affect experiment results and research conclusions.