Philipp Richter

Distilling the Internet's Application Mix from Packet-Sampled Traffic

As the Internet continues to grow both in size and in terms of the volume of traffic it carries, more and more networks in the different parts of the world are relying on an increasing number of distinct ways to exchange traffic with one another. As a result, simple questions such as “What is the application mix in today’s Internet?” may produce non-informative simple answers unless they are refined by specifying the vantage point where the traffic is observed, the networks that are involved, or even the type of interconnection used.

A Primer on IPv4 Scarcity

With the ongoing exhaustion of free address pools at the registries serving the global demand for IPv4 address space, scarcity has become reality. Networks in need of address space can no longer get more address allocations from their respective registries. In this work we frame the fundamentals of the IPv4 address exhaustion phenomena and connected issues. We elaborate on how the current ecosystem of IPv4 address space has evolved since the standardization of IPv4, leading to the rather complex and opaque scenario we face today. We outline the evolution in address space management as well as address space use patterns, identifying key factors of the scarcity issues. We characterize the possible solution space to overcome these issues and open the perspective of address blocks as virtual resources, which involves issues such as differentiation between address blocks, the need for resource certification, and issues arising when transferring address space between networks.

Lost in Space: Improving Inference of IPv4 Address Space Utilization

One challenge in understanding the evolution of the Internet infrastructure is the lack of systematic mechanisms for monitoring the extent to which allocated IP addresses are actually used. In this paper, we advance the science of inferring IPv4 address space utilization by proposing a novel taxonomy and analyzing and correlating results obtained through different types of measurements. We have previously studied an approach based on passive measurements that can reveal used portions of the address space unseen by active approaches. In this paper, we study such passive approaches in detail, extending our methodology to new types of vantage points and identifying traffic components that most significantly contribute to discovering used IPv4 network blocks. We then combine the results we obtained through passive measurements together with data from active measurement studies, as well as measurements from Border Gateway Protocol and additional data sets available to researchers. Through the analysis of this large collection of heterogeneous data sets, we substantially improve the state of the art in terms of: 1) understanding the challenges and opportunities in using passive and active techniques to study address utilization and 2) knowledge of the utilization of the IPv4 space.

Beyond Counting: New Perspectives on the Active IPv4 Address Space

In this study, we report on techniques and analyses that enable us to capture Internet-wide activity at individual IP address-level granularity by relying on server logs of a large commercial content delivery network (CDN) that serves close to 3 trillion HTTP requests on a daily basis. Across the whole of 2015, these logs recorded client activity involving 1.2 billion unique IPv4 addresses, the highest ever measured, in agreement with recent estimates. Monthly client IPv4 address counts showed constant growth for years prior, but since 2014, the IPv4 count has stagnated while IPv6 counts have grown. Thus, it seems we have entered an era marked by increased complexity, one in which the sole enumeration of active IPv4 addresses is of little use to characterize recent growth of the Internet as a whole. With this observation in mind, we consider new points of view in the study of global IPv4 address activity. Our analysis shows significant churn in active IPv4 addresses: the set of active IPv4 addresses varies by as much as 25% over the course of a year. Second, by looking across the active addresses in a prefix, we are able to identify and attribute activity patterns to networkm restructurings, user behaviors, and, in particular, various address assignment practices. Third, by combining spatio-temporal measures of address utilization with measures of traffic volume, and sampling-based estimates of relative host counts, we present novel perspectives on worldwide IPv4 address activity, including empirical observation of under-utilization in some areas, and complete utilization, or exhaustion, in others.

A Multi-perspective Analysis of Carrier-Grade NAT Deployment

As ISPs face IPv4 address scarcity they increasingly turn to network address translation (NAT) to accommodate the address needs of their customers. Recently, ISPs have moved beyond employing NATs only directly at individual customers and instead begun deploying Carrier-Grade NATs (CGNs) to apply address translation to many independent and disparate endpoints spanning physical locations, a phenomenon that so far has received little in the way of empirical assessment. In this work we present a broad and systematic study of the deployment and behavior of these middleboxes. We develop a methodology to detect the existence of hosts behind CGNs by extracting non-routable IP addresses from peer lists we obtain by crawling the BitTorrent DHT. We complement this approach with improvements to our Netalyzr troubleshooting service, enabling us to determine a range of indicators of CGN presence as well as detailed insights into key properties of CGNs. Combining the two data sources we illustrate the scope of CGN deployment on todays Internet, and report on characteristics of commonly deployed CGNs and their effect on end users.

Back-Office Web Traffic on The Internet

Although traffic between Web servers and Web browsers is readily apparent to many knowledgeable end users, fewer are aware of the extent of server-to-server Web traffic carried over the public Internet. We refer to the former class of traffic as front-office Internet Web traffic and the latter as back-office Internet Web traffic (or just front-office and back-office traffic, for short). Back-office traffic, which may or may not be triggered by end-user activity, is essential for todays Web as it supports a number of popular but complex Web services including large-scale content delivery, social networking, indexing, searching, advertising, and proxy services. This paper takes a first look at back-office traffic, measuring it from various vantage points, including from within ISPs, IXPs, and CDNs. We describe techniques for identifying back-office traffic based on the roles that this traffic plays in the Web ecosystem. Our measurements show that back-office traffic accounts for a significant fraction not only of core Internet traffic, but also of Web transactions in the terms of requests and responses. Finally, we discuss the implications and opportunities that the presence of back-office traffic presents for the evolution of the Internet ecosystem.

Inferring BGP blackholing activity in the internet

The Border Gateway Protocol (BGP) has been used for decades as the de facto protocol to exchange reachability information among networks in the Internet. However, little is known about how this protocol is used to restrict reachability to selected destinations, e.g., that are under attack. While such a feature, BGP blackholing, has been available for some time, we lack a systematic study of its Internet-wide adoption, practices, and network efficacy, as well as the profile of blackholed destinations. In this paper, we develop and evaluate a methodology to automatically detect BGP blackholing activity in the wild. We apply our method to both public and private BGP datasets. We find that hundreds of networks, including large transit providers, as well as about 50 Internet exchange points (IXPs) offer blackholing service to their customers, peers, and members. Between 2014--2017, the number of blackholed prefixes increased by a factor of 6, peaking at 5K concurrently blackholed prefixes by up to 400 Autonomous Systems. We assess the effect of blackholing on the data plane using both targeted active measurements as well as passive datasets, finding that blackholing is indeed highly effective in dropping traffic before it reaches its destination, though it also discards legitimate traffic. We augment our findings with an analysis of the target IP addresses of blackholing. Our tools and insights are relevant for operators considering offering or using BGP blackholing services as well as for researchers studying DDoS mitigation in the Internet.

Understanding the Share of IPv6 Traffic in a Dual-Stack ISP

After almost two decades of IPv6 development and consequent efforts to promote its adoption, the current global share of IPv6 traffic still remains low. Urged by the need to understand the reasons that slow down this transition, the research community has devoted much effort to characterize IPv6 adoption, i.e., if ISPs and content providers enable IPv6 connectivity. However, little is known about how much the available IPv6 connectivity is actually used and precisely which factors determine whether data is exchanged over IPv4 or IPv6. To tackle this question, we leverage a relevant vantage point: a dual-stack residential broadband network. We study interactions between applications, devices, equipment and services, and illustrate how these interactions ultimately determine the IPv6 traffic share. Lastly, we elaborate on the potential scenarios that dual-stack ISPs and content providers may confront during the Internet’s transition to IPv6.

Scanning the Internet for Liveness

Internet-wide scanning depends on a notion of liveness: does a target IP address respond to a probe packet? However, the interpretation of such responses, or lack of them, is nuanced and depends on multiple factors, including: how we probed, how different protocols in the network stack interact, the presence of filtering policies near the target, and temporal churn in IP responsiveness. Although often neglected, these factors can significantly affect the results of active measurement studies. We develop a taxonomy of liveness which we employ to develop a method to perform concurrent IPv4 scans using ICMP, five TCP-based, and two UDP-based protocols, comprehensively capturing all responses to our probes, including negative and cross-layer responses. Leveraging our methodology, we present a systematic analysis of liveness and how it manifests in active scanning campaigns, yielding practical insights and methodological improvements for the design and the execution of active Internet measurement studies.

Empirical analysis of the effects and the mitigation of IPv4 address exhaustion

IP addresses are essential resources for communication over the Internet. In IP version 4, an address is represented by 32 bits in the IPv4 header; hence there is a finite pool of roughly 4B addresses available. The Internet now faces a fundamental resource scarcity problem: The exhaustion of the available IPv4 address space. In 2011, the Internet Assigned Numbers Authority (IANA) depleted its pool of available IPv4 addresses. IPv4 scarcity is now reality. In the subsequent years, IPv4 address scarcity has started to put substantial economic pressure on the networks that form the Internet. The pools of available IPv4 addresses are mostly depleted and today network operators have to find new ways to satisfy their ongoing demand for IPv4 addresses. Mitigating IPv4 scarcity is not optional, but mandatory: Networks facing address shortage have to take action in order to be able to accommodate additional subscribers and customers. Thus, if not confronted, IPv4 scarcity has the potential to hinder further growth of the Internet. Addressing is a collective and global effort, and interconnectivity among networks forms the very basis of the Internet. At the same time, the decentralized nature of the Internet and independent decisions by its different stakeholders create a complex and opaque problem space. Different approaches to mitigating IPv4 address scarcity in the short and in the long term exist. The solution space includes increasing the utilization of already available IPv4 address resources, introducing IPv4 address multiplexing techniques, incrementally transitioning to IPv6 (but maintaining IPv4 compatibility), or purchasing IPv4 address space on address markets. Individual network operators make independent decisions on which approach—or combination of approaches—to pursue. Each option has different ramifications, benefits, and consequences for the individual networks and their connected end users. At the same time, the increasing and disparate deployment of different mitigation techniques and the coexistence of two addressing protocols (IPv4 and IPv6) adds heterogeneity to the network and hence changes the overall connectivity and communication structure of the Internet. In spite of the pressing relevance of the topic, we lack a comprehensive understanding of IPv4 address exhaustion and its effects, as well as of the pervasiveness, impact, and ramifications of the different mitigation strategies. This dissertation provides a systematic empirical analysis of the phenomenon of IPv4 address space exhaustion, its effect on the Internet as a whole, and its stakeholders individually. We first provide an empirical lay-of-the-land of the history and the current status of the IPv4 address space and illuminate the interplay between policy and governance decisions and address use. We then develop techniques that allow us to measure the potential, the pervasiveness, and the ramifications of the individual mitigation strategies that network operators may choose to pursue. In particular, we measure global IPv4 address activity patterns, which allows us to both study exhaustion effects on address activity and to assess the potential for increasing the utilization of the IPv4 address space. We develop tools to detect CarrierGrade NAT (CGN) presence on the Internet at scale, and identify dominant properties of deployed CGN instances and their respective impact. Lastly, we examine aspects of IPv6 adoption, where we measure inter-domain connectivity and traffic carried over IPv4 and IPv6, and interactions between IPv4 and IPv6 traffic in detail, allowing us to pinpoint barriers and challenges for IPv6 adoption. We strive to both illuminate the broader impact of address exhaustion on the Internet and its structure as well as to provide practical insights to support the ongoing process of mitigating address scarcity. Our results can serve as a basis for network operators and policymakers to make informed decisions on how to approach IPv4 address exhaustion. They may also inform future measurement studies and the design of operational systems, which need to adapt to this increasingly complex environment.