Nevil Brownlee

Comparison of public end-to-end bandwidth estimation tools on high-speed links

In this paper we present results of a series of bandwidth estimation experiments conducted on a high-speed testbed at the San Diego Supercomputer Center and on OC-48 and GigE paths in real world networks. We test and compare publicly available bandwidth estimation tools: abing, pathchirp, pathload, and Spruce. We also tested Iperf which measures achievable TCP throughput. In the lab we used two different sources of known and reproducible cross-traffic in a fully controlled environment. In real world networks we had a complete knowledge of link capacities and had access to SNMP counters for independent cross-traffic verification. We compare the accuracy and other operational characteristics of the tools and analyze factors impacting their performance.

DNS measurements at a root server

The Domain Name System (DNS) prescribes domain names to be used in network transactions (email, web requests, etc.) instead of IP addresses. The root of the DNS distributed database is managed by 13 root nameservers. We passively measure the performance of one of them: F.root-servers.net. These measurements show an astounding number of bogus queries: from 60-85% of observed queries were repeated from the same host within the measurement interval. Over 14% of a root servers query load is due to queries that violate the DNS specification. Denial of service attacks using root servers are common and occurred throughout our measurement period (7-24 Jan 2001). Though not targeted at the root servers, DOS attacks often use root servers as reflectors toward a victim network. We contrast our observations with those found in an earlier study of DNS root server performance by Danzig et. al., (1992).

Passive Monitoring of DNS Anomalies

We collected DNS responses at the University of Auckland Internet gateway in an SQL database, and analyzed them to detect unusual behaviour. Our DNS response data have included typo squatter domains, fast flux domains and domains being (ab)used by spammers. We observe that current attempts to reduce spam have greatly increased the number of A records being resolved. We also observe that the data locality of DNS requests diminishes because of domains advertised in spam.

Measurements and Laboratory Simulations of the Upper DNS Hierarchy

Given that the global DNS system, especially at the higher root and top-levels, experiences significant query loads, we seek to answer the following questions: (1) How does the choice of DNS caching software for local resolvers affect query load at the higher levels? (2) How do DNS caching implementations spread the query load among a set of higher level DNS servers? To answer these questions we did case studies of workday DNS traffic at the University of California San Diego (USA), the University of Auckland (New Zealand), and the University of Colorado at Boulder (USA). We also tested various DNS caching implementations in fully controlled laboratory experiments. This paper presents the results of our analysis of real and simulated DNS traffic. We make recommendations to network administrators and software developers aimed at improving the overall DNS system.

Passive measurement of one-way and two-way flow lifetimes

Flow based analysis has been considered a simple and effective approach in network analysis. 5-tuple (unidirectional) flows are used in many network traffic, however, often these analyses require bidirectional packet matching to observe the interactions. Separating the flows into two categories as one-way (packets in one direction only) and two-way (packets in both directions) flows can yield further insight. We have examined traces of Auckland traffic for 2000, 2003 and 2006, and analyzed their one-way and two-way flows. We observed several behaviors and the changes in flow sizes and their lifetimes over time. In our traces, we observe that one-way flows are mostly malicious, re-transmissions, and some are long-lived. Two-way flows are mostly normal end-to-end transmissions with their lifetimes/RTTs decreasing, their sizes increasing, and many short-lived flows mostly depict errors in TCP. Also, we observe similarity between one-way and two-way flow sizes for their lifetimes.

Observations of UDP to TCP Ratio and Port Numbers

Widely used protocols (UDP and TCP) are observed for variations of the UDP to TCP ratio and of port number distribution, both over time and between different networks. The purpose of the study was to understand the impact of application trends, especially the growth in media streaming, on traffic characteristics. The results showed substantial variability but little sign of a systematic trend over time, and only wide spreads of port number usage.

Two days in the life of the DNS anycast root servers

The DNS root nameservers routinely use anycast in order to improve their service to clients and increase their resilience against various types of failures. We study DNS traffic collected over a two-day period in January 2006 at anycast instances for the C, F and K root nameservers. We analyze how anycast DNS service affects the worldwide population of Internet users. To determine whether clients actually use the instance closest to them, we examine client locations for each root instance, and the geographic distances between a server and its clients. We find that frequently the choice, which is entirely determined by BGP routing, is not the geographically closest one. We also consider specific AS paths and investigate some cases where local instances have a higher than usual proportion of non-local clients. We conclude that overall, anycast roots significantly localize DNS traffic, thereby improving DNS service to clients worldwide.

A study of burstiness in TCP flows

We study the burstiness of TCP flows at the packet level. We aggregate packets into entities we call “flights”. We show, using a simple model of TCP dynamics, that delayed-acks and window dynamics would potentially cause flights at two different timescales in a TCP flow— the lower at the order of 5-10 ms (sub-RTT) and the higher at about 10 times this value (order of an RTT seen by the flow). The model suggests that flight sizes would be small at the lower timescale, regardless of the network environment. The model also predicts that the network conditions required for the occurrence of flights at the larger timescale are either large buffers or large available bandwidths — both of which result in a high bandwidth delay product environment. We argue that these two conditions indicate that the TCP flow does not operate in a congestion control region , either because the source of traffic is unaware of congestion or because there is so much bandwidth that congestion control is not required. We verify our model by passive Internet measurement. Using the trace files obtained, we collect statistics on flights at the two timescales in terms of their frequency and size. We also find the dependence of the sizes and frequency of flights on the Internet environment in which they occurred. The results concur strongly with our hypothesis on the origins of flights, leading us to the conclusion that flights are effective indicators of excess resource in the Internet.

Using NeTraMet for production traffic measurement

When monitoring network traffic, a flow measurement approach can provide the advantage of data reduction in near real-time. RTFM, the IETFs standard, generalised architecture for measuring traffic flows, and NeTraMet, an open-source implementation of that architecture, are introduced. The results of a NeTraMet usage survey are presented, together with a detailed description of the NeTraMet components and the ways they can be used to construct traffic flow measurement systems for any particular network. Nifty, an X Window near-real-time traffic flow analyser, is also described. Experiences with NeTraMet are summarised, highlighting its usefulness in building customised flow measurement systems.

One-way traffic monitoring with iatmon

During the last decade, unsolicited one-way Internet traffic has been used to study malicious activity on the Internet. Researchers usually observe such traffic using network telescopes deployed on darkspace (unused address space). When darkspace observations began ten years ago, one-way traffic was minimal. Over the last five years, however, traffic levels have risen so that they are now high enough to require more subtle differentiation --- raw packet and byte or even port counts make it hard to discern and distinguish new activities. To make changes in composition of one-way traffic aggregates more detectable, we have developed iatmon (Inter-Arrival Time Monitor), a freely available measurement and analysis tool that allows one to separate one-way traffic into clearly-defined subsets. Initially we have implemented two subsetting schemes; source types, based on the schema proposed in [12]; and inter-arrival-time (IAT) groups that summarise source behaviour over time. We use 14 types and 10 groups, giving us a matrix of 140 type+group subsets. Each subset constitutes only a fraction of the total traffic, so changes within the subsets are easily observable when changes in total traffic levels might not even be noticeable. We report on our experience with this tool to observe changes in one-way traffic at the UCSD network telescope over the first half of 2011. Daily average plots of source numbers and their traffic volumes show clear long-term changes in several of our types and groups.