Flexsealing BGP Against Route Leaks: Peerlock Active Measurement and Analysis
FFlexsealing BGP Against Route Leaks:Peerlock Active Measurement and Analysis
Tyler McDaniel, Jared M. Smith, and Max Schuchard
University of Tennessee, Knoxville{bmcdan16, jms, mschucha}@utk.edu
Abstract —BGP route leaks frequently precipitate serious dis-ruptions to inter-domain routing. These incidents have plaguedthe Internet for decades while deployment and usability issuescripple efforts to mitigate the problem. Peerlock, presented in2016, addresses route leaks with a new approach. Peerlockenables filtering agreements between transit providers to protecttheir own networks without the need for broad cooperationor a trust infrastructure. We outline the Peerlock system andone variant, Peerlock-lite, and conduct live Internet experi-ments to measure their deployment on the control plane. Ourmeasurements find evidence for significant Peerlock protectionbetween Tier 1 networks in the peering clique, where 48% ofpotential Peerlock filters are deployed, and reveal that manyother networks also deploy filters against Tier 1 leaks. Toguide further deployment, we also quantify Peerlock’s impacton route leaks both at currently observed levels and underhypothetical future deployment scenarios via BGP simulation.These experiments reveal present Peerlock deployment restrictsTier 1 leak export to 10% or fewer networks for 40% of simulatedleaks. Strategic additional Peerlock-lite deployment at all largeISPs (<1% of all networks), in tandem with Peerlock within thepeering clique as deployed, completely mitigates about 80% ofsimulated Tier 1 route leaks.
I. I
NTRODUCTION
The Internet consists of many
Autonomous Systems (ASes)with distinct IP prefixes, routing policies, and inter-AS con-nections. These networks exchange routes with neighboringASes over the control plane to connect hosts in disparateASes and create the illusion for users of a single, unifiedInternet. Unfortunately, there are few security controls on routeexchange. ASes behaving adversarially, whether intentionallyor by mistake, can export routes that should be kept internallyor shared with only a subset of their neighbors. Because thelanguage ASes use to communicate - the
Border GatewayProtocol or BGP - does not package validation informationwith routes, remote networks often receive and propagate these route leaks throughout the control plane. Leaks frequentlysteer user traffic on the data plane onto unintended paths thatlack capacity for the additional traffic. The end result is soaringlatency or complete availability loss for destination services.Recent route leaks to prefixes hosting major content/service providers like Spotify [21], Cloudflare [23] and Google [11]have highlighted the global impact of this problem.Existing tools designed to curtail leaks, like the manyInternet Routing Registries (IRRs), are challenging to deployor limited in scope. IRRs are databases where ASes canpublish their routing policies. Other ASes can then convertIRR-stored policies into filters to validate received routes. IRR-based filtering is limited by its requirement for broad ASparticipation, however, as the motivations and sophisticationof network operators varies greatly between ASes [22]. OtherBGP security extensions, like the Resource Public Key Infras-tructure (RPKI), only enable filtering for a subset of leaks (e.g.re-origination leaks for RPKI).The
Peerlock [43], [18] leak defense system was presentedin 2016 to address the need for a deployable solution. EachPeerlock deployment occurs between two neighboring ASes,the protector
AS and protected
AS. The protector AS agreesto filter routes that transit the protected AS unless they arrivedirectly from the protected AS or one of its designatedupstreams. The filter prevents the protector AS from propa-gating or steering its traffic onto any leaked route that transitsthe protected AS, regardless of origin AS/destination prefix.Peerlock is designed to leverage the rich web of relationshipsthat exist between transit networks in the Internet’s core, andfunctions without coordination with other ASes on potentialleak paths. This makes Peerlock especially viable in the peering clique formed by the 19
Tier 1
ASes that sit atop theinter-domain routing hierarchy. A related technique, Peerlock-lite, enables networks to spot likely leaks without prior out-of-band communication. ASes deploying Peerlock-lite droproutes arriving from customers that contain a Tier 1 AS; itis highly improbable that customers are providing transit forlarge global networks.Our first contribution is a measurement ofPeerlock/Peerlock-lite deployment on the control plane.In Section IV we design, execute, and evaluate active Internetmeasurements to search for evidence of filtering consistentwith these systems. Our experiments use BGP poisoning, atechnique used in prior work for traffic engineering [41] andpath discovery [1], to mimic route leaks that transit sometarget AS. We then listen for which networks propagate -or filter - these "leaks" relative to control advertisements.This information feeds several inference techniques designedto uncover which ASes are Peerlocking for (protecting) thetarget AS. a r X i v : . [ c s . N I] A ug otably, we find substantial Peerlock deployment within thepeering clique: about 48% of possible filtering rules (153/342)are already implemented within this set. Further, many non-Tier 1 ASes - including nearly 40% of large ISPs observedduring our experiments - perform some Peerlock-lite filteringon Tier 1 AS leaks. Evidence for Peerlock filtering of non-Tier1 leaks is virtually nonexistent, though three Tier 1 networks(AS 12956, AS 2914, and AS 3320) each filter leaks for morethan 20 non-Tier 1 ASes.After detecting current Peerlock/Peerlock-lite deployments,we ask how well these systems mitigate Tier 1 leaks. Internet-scale BGP simulations in Section V test over 6,000 simulatedTier 1 leaks against observed Peerlock/Peerlock-lite levels toquantify the effect of these systems as deployed. We testthe same leaks against six hypothetical extended deploymentscenarios to understand where additional filters should beplaced to isolate leaks.We find that Peerlock filtering within the peering clique ishelpful, but not sufficient to mitigate Tier 1 route leaks onits own. Consistent with prior work on BGP filtering [8], ourexperiments show that positioning filters at relatively few ASes- the roughly 600 large ISPs - can play a decisive role in leakprevention. About 80% of simulated leaks were completelymitigated by uniform Peerlock-lite filter deployment at largeISPs, with fewer than 10% of leaks spreading beyond 10%of the topology. These figures are especially encouragingbecause Peerlock-lite is based on a simple route validity checkinformed by the valley-free routing model [7] that requires noout-of-band communication.In this paper, we make the following key contributions: • We give an overview of the Peerlock and Peerlock-lite filtering systems, and consider their benefits andlimitations relative to existing tools in Section III. • We describe how we adapt existing Internet measurementtechniques to probe Peerlock/Peerlock-lite deployment onthe control plane and introduce a novel inference methodin Section IV-A. • We actively measure where Peerlock and Peerlock-litefilters are deployed with PEERING [35] and CAIDA’sBGPStream [33] in Section IV-B, with a discussion ofresults in Section IV-C. • We simulate thousands of Tier 1 route leaks againstseveral protection scenarios in Section V-B, and presenta new path encoding method to understand how thesescenarios influence leak propagation and export in Sec-tion V-C. II. B
ACKGROUND
A. The Border Gateway Protocol
The Internet is a confederation of about 69,000 smallernetworks, called
Autonomous Systems or ASes . ASes exchangerouting information via the Border Gateway Protocol (BGP)to enable global connectivity. Each AS originates routes to itshosted prefixes; these routes are advertised to neighbors via
BGP updates . Each update contains a prefix and a collectionof other attributes, including an AS PATH that describes the route’s AS-level hops. ASes compare all received updates viathe BGP decision process to select a single best path to everydestination prefix. Both path qualities (like AS PATH length)and local network policies (e.g., business relationship withadvertising AS) are taken in account when selecting a bestpath, but policies take precedence in the process. Once an ASselects a best path for a given prefix, it prepends its uniqueAS number (ASN) to the path and advertises only that path toits neighbors.Paths learned from customer ASes - those purchasingtransit - are advertised to all connections. Provider-learnedroutes, meanwhile, are generally only advertised to an AS’scustomers. Peer ASes exchange traffic without compensation,and likewise advertise routes learned from one another onlyto customer ASes. Limitations on non-customer learned routeexport prevents customer ASes from transiting traffic betweenpeers/providers at their own expense. This dynamic, known asthe Gao-Rexford or valley-free routing model [7], guides theexchange of routes on the control plane. No widely-deployedmechanism enforces this model, but the economic incentivesit describes shape AS path export behavior.The customer cone [10] is one product of this model. AnAS’s customer cone is the set of all ASes reachable from theAS via only provider to customer links. Stated simply, theseare the AS’s direct and indirect customers. Customer conesize is one of the few publicly observable features commonlyused to judge an AS’s influence on the control plane, e.g.in CAIDA’s AS ranking [49]. Customer cone size is thebasis for the UCLA classification presented in [32] widelyused in research on this topic [40], [48], [1], [4], [52]. Thisscheme separates ASes into 1) Tier 1 ASes, who have noproviders, form a peering clique, and can transit traffic to anyprefix without compensation, 2) large ISPs with more than50 customer cone ASes, 3) small ISPs with 5-50 customercone ASes, and 4) stub ASes with fewer than 5 direct/indirectcustomers.
B. Route Leaks
Despite its vital role in binding together Internet networks,BGP is missing key security features like cryptographic hard-ening of routes exchanged between ASes or trusted certifi-cation binding ASes to owned prefixes. This leads to twocommon classes of major inter-domain routing mishaps, pre-fix hijacking and route leaks . Prefix hijacks occur when anetwork, often unintentionally, originates or advertises a fakebut attractive (e.g. shorter or more specific) route to prefixesowned by another AS. Traffic destined for those prefixes isthen intercepted by the hijacker. A number of recent studiesfocus on hijack mitigation [38], [34], [51].Route leaks are defined in RFC 7908 as the propagationof an advertisement beyond its intended scope [44]. Type 1-4 leaks all cover various valley-free routing violations, i.e.advertising one peer/provider’s routes to another peer/provider.Because remote ASes have little or no information on rela-tionships between non-neighboring networks, they generallycannot distinguish leaks from valid routes, and may propagate2hem throughout the topology. Type 5 leaks occur when oneprovider’s routes are announced to another with the AS PATHstripped, effectively re-originating the prefix from the leaker.Finally, a Type 6 leak involves an AS announcing routes usedinternally to its neighbors. These routes are often more specificthan externally announced routes; this makes the leaks moreattractive in the BGP decision process and encourages theirspread to other remote networks.Globally disruptive route leaks occur frequently [21], [39],[11], [27], [53], [26]. The 2019 Verizon-Cloudflare leak [23]is a high profile example. A small ISP, AS 33154, leakedspecific internal prefixes (Type 6) to Cloudflare and manyother destinations to its customer, AS 396531. AS 396531committed a Type 1 leak by advertising this route to itsother provider, AS 701 Verizon. Verizon propagated the leak,which spread widely on the control plane because it wasmore specific than legitimate available routes (see depiction inFig. 1). Traffic for Cloudflare, a leading content distributionnetwork (CDN), was funneled through small networks. Manyof the thousands of websites and services backed by Cloud-flare suffered degraded service until the leak was resolvedvia out-of-band communication between Cloudflare and AS33154 [47]. eFig. 1: 2019 Verizon/Cloudflare leak. Other destination ser-vices were also affected.
C. Route Leak Prevention
There are a number of tools available to assist networkoperators in preventing route leaks. The Resource Public KeyInfrastructure [25] is a trusted repository for certificates thatbind IP prefixes to owning ASes’s public keys, called RouteOrigin Authorizations (ROAs). Remote networks can validateBGP updates against ROAs in the RPKI, a process calledRoute Origin Validation (ROV). Widespread ROV filteringcould prevent Type 5 (and some Type 6) leaks and many prefixhijacking attacks. Unfortunately, ROA/ROV deployment hassuffered from circular deployment logic; it is meaningless fororigin ASes to invest in publishing ROAs until ROV is widelyimplemented, but ROV is ineffective without ROAs. This issuehas been identified as a major obstacle to ROV deployment [8],[12]. NIST estimates that just 20% of prefixes are covered bya valid ROA [30]. Internet routing registries (IRRs) back another leak pre-vention system. IRRs are databases where AS operators canstore their routing policies. Remote networks can ingest thesepolicies to inform filters that block unintended/invalid ad-vertisements. IRR databases are operated by private firms,regional Internet registries, and other interests [15], and policyentries are often mirrored between them. A complete, up-to-date IRR would eliminate many Type 1-4 route leaks.Like ROV filtering, though, IRR filtering is hampered bydeployment headaches. ASes’ routing policies are interdepen-dent, so changes to one network’s stored policies can rendermany others obsolete. Operators in smaller, resource-limitednetworks can avoid periodic updates by configuring permissiverouting policies; large transit ASes have complex, dynamicrouting policies that require frequent changes to dependentnetworks’ filters [22]. These issues, combined with poor ornon-existent authentication, have resulted in inconsistent andout-of-date IRRs. Though leading organizations like RIPEhave launched efforts to improve IRR quality [29], operatorincentive and dependency issues will continue to limit theirusefulness.Other filtering techniques include max-prefix limit filtering,where a network caps the number of prefixes it will acceptfrom a neighbor. This prevents mass prefix spills like the 2017Level 3 leak [27], but not more targeted (yet highly disruptive)leaks like the Verizon/Cloudflare incident described earlier.BGPSec [24] is a protocol extension for cryptographic ASpath hardening. This would prevent some types of hijacking,but BGPSec has not been commercially implemented and isnot designed to prevent route leaks.Finally, a communities-based "down-only" (DO) leak solu-tion has been proposed [45]. Large BGP communities [42] aresignals containing three integers that can be attached to routes.The DO system relies on providers/peers marking a route"down-only" using the first two integers in a large community,with their ASN included as the third integer, before passing theroute to customers or peers. If these customers/peers attemptto re-advertise the route to other providers/peers, the attachedDO community will clearly signal a route leak. While thissystem would prevent many leaks if properly implemented,it relies on customers/peers preserving DO information whenpropagating advertisements. Moreover, some leaks - like theinternal route leaks in the Verizon/Cloudflare incident - wouldnot be arrested by this system.
D. BGP PoisoningBGP poisoning is a technique designed to manipulate theBGP decision process in remote networks. ASes originating aprefix can poison an advertisement by including the ASNsof remote networks in the AS PATH. Often, the poisonedASNs will be inserted between copies of the origin’s ASN.This "sandwiching" ensures traffic is routed properly and thatthe advertisement is valid for ROV filtering purposes (seeFig. 2). BGP prevents cycles from forming in the topologyby requiring ASes to drop routes containing their own ASNin the AS PATH; this is known as BGP loop detection. So,3ll networks included in the poisoned update’s AS PATH -the poisoned
ASes - will filter it. Poisons can be used forinbound traffic engineering purposes [41], [40], [20] but weemploy them in Section IV to mimic route leaks transiting thepoisoned AS.Fig. 2: BGP poisoning. AS 1 originates a route with AS 2prepended to path (left); AS 2 filters the update (center), butAS 3 propagates (right).III. T HE P EERLOCK S YSTEM
The challenge of leak filtering stems from the topologicalscope asymmetry between BGP routes and the perspectiveof individual AS operators who evaluate them. Routes spanthe topology (global scope); operators only know their ownrelationships with adjoining ASes (local scope). Filteringsystems built on the RPKI [13] and IRRs [14] partially addressthis asymmetry by applying additional information to theroute evaluation process. However, these existing solutionshave limitations that have hamstrung their deployment. Mostcritically, their effectiveness depends on the cooperation ofmany unincentivized remote ASes as detailed in Section II-C.
A. PeerlockPeerlock , first detailed by NTT in 2016 [43], [18], is aleak filtering scheme based on out-of-band information ex-change between BGP neighbors. Peerlock requires a single AS(the protected AS ) to designate authorized upstreams to theirBGP neighbor (the protector AS ). This communication dis-tributes AS relationship information between peers to decreaseroute/filterer scope asymmetry. The protector AS then rejectsany BGP update whose AS PATH contains the protected ASunless received 1) directly from the protected AS, or 2) froman authorized upstream, with the protected AS immediatelyfollowing the authorized upstream in the AS PATH. We saythat the protector AS is Peerlocking for the protected AS.See Fig. 3 for a depiction of the system. In this paper, wewill often refer to a single instance of Peerlock - that is, oneprotector/protected pairing - as a Peerlock rule .Here we describe Peerlock’s benefits and drawbacksrelative to previous leak prevention systems, each of whichis described in detail in Section II-C. These comparisons aresummarized in Table 1.
RPKI/ROV Comparison : Peerlock provides broader leaktype coverage than RPKI/ROV filtering without a trustinfrastructure requirement. However, Peerlock only applies toleaks that violate configured topological rules (Types 1-4), so Fig. 3: Simple Peerlock deployment. Protector AS filtersupdates containing the peer Protected AS from unauthorizedpropagators.Type 5 (re-origination) and Type 6 (internal route) leaks falloutside its scope. ROAs tie prefixes to valid originating ASes,so ROV filtering can prevent Type 5 leaks. Additionally,ROAs can be configured with a max prefix length to preventsome internal route leaks and hijacks, although recent workhas identified vulnerabilities in this feature [9]. BecausePeerlock and RPKI/ROV filtering cover different leak types,Peerlock is complementary to ROV filtering rather than areplacement.
IRR Comparison : IRRs are policy object databases capableof storing participating networks’ routing intentions with greatdetail and fine granularity (prefix level). Any AS wishing toenforce these intentions can automatically derive filters fromstored objects using software tools, whereas Peerlock ruleconfiguration requires setup between each protector/protectedAS pair. Unfortunately, IRRs suffer from incentive misalign-ment, governance, and rule dependency issues as described inSection II-C. Peerlock rules are self-contained, and changes donot affect other rules. This encapsulation avoids the cascadingdependency problem exhibited by IRRs, where one AS’spolicy changes may render many other AS’s entries obsolete.Most importantly, Peerlock allows the protector AS tostop leaks that transit the protected network regardless ofthe actions of ASes along potential leak paths; the value ofIRR-based filters depends on many remote networks to storeaccurate policy entries. Peerlock’s relatively light cooperationrequirement only requires that ASes with an existingrelationship communicate information between themselves.This dynamic enables the best resourced, positioned, andincentivized networks (i.e., those serving the most customers)to block route leak propagation regardless of other remotenetworks’ actions.
Max Prefix Comparison : BGP’s max prefix feature enablesnetworks to limit the number of prefixes they will accept4 ystem Coverage Requirements Notes
RPKI/ROV Type 5 internal leak,Type 6 re-origination RPKI trust infrastructure,local ROA registration &remote ROV checks Type 5 coverage depends on optional ROA max lengthIRR Filtering Potentially all leak types;depends on stored policyobject specificity. Correct, fresh policy objects &derived filters along potential leakpath Quality issues, misaligned incentivesMax Prefix All leak types Filter with meaningful max prefixlimit somewhere on potential leakpath Only effective when many prefixes are leaked
Table 1: Common leak filtering systems.over each neighboring AS connection. Mass route leaks- those involving many prefixes - are filtered once prefixvolume over an inter-AS link exceeds the preset limit. Maxprefix filtering affords broad protection across leak types, butcannot stop leaks involving few (potentially critical/highlytrafficked) prefixes. On the other hand, Peerlock cannot stopleaks that do not violate established topological constraintsregardless of volume, but is effective against more selectiveleaks unprotected by max prefix limits.
Other Considerations : Currently, each Peerlock rule mustbe manually configured, although at least one method hasbeen proposed to automate Peerlock [16]. Peerlock alsolacks a standard to describe how out-of-band informationis exchanged between participants. Without a detailed andsecure protocol for rule configuration, Peerlock is vulnerableto exploitation; fraudulent rules affect route export, and couldbe used to engineer traffic flows. Furthermore, operators mustdefine their own ad-hoc protocols for communicating rulesthat may not guarantee authenticity and/or confidentiality.Virtually all leak solutions discussed here, including IRR,RPKI/ROV, and AS PATH filtering, are recommended by bestpractices group Mutually Agreed Norms for Routing Security(MANRS) [5].
B. Peerlock-litePeerlock-lite [18] (or Tier 1 filter, "big networks" filter)is a related technique, based on the assumption that transitproviders should never receive a route whose AS PATH in-cludes a Tier 1 AS from a customer. This is a valid assumptionunder the valley-free routing model [7], because such anupdate implies the customer is providing transit for the Tier 1AS; otherwise, the customer would not export (leak) the routeto another provider. However, Tier 1 ASes have no providersby definition. This logic can be extended heuristically to anyother large non-Tier 1 networks that the provider does notexpect the customer to export.This simple logic yields an equally simple filtering rulefor transit providers - reject any updates from customers thatcontain a Tier 1/large transit ASN. See Fig. 4 for a depiction ofthis filtering technique. Peerlock-lite filters are limited to Tier1/large transit provider leaks, but they require no out-of-bandinformation to configure. Moreover, Tier 1 ASes’ position atthe Internet’s core results in their frequent presence on AS PATHs of highly disruptive leaks, e.g. the Verizon/Cloudflareleak [23] and the Enzu/AWS/Spotify leak [21].Fig. 4: Example Peerlock-lite deployment. Provider AS filtersupdates from its customer that include a Tier 1 AS.IV. M
EASURING P EERLOCK D EPLOYMENT
Our initial experiments seek to establish the current state ofPeerlock deployment on the control plane. As discussed in theprevious section, every Peerlock rule is configured between apair of networks: the protector AS and the protected AS. Eachof the experiments in this section works to identify some orall Peerlock/Peerlock-lite protectors for a targeted AS.
A. Measurement Methodology
Experimental Design:
Each set of measurement experimentsin this section is designed to discover Peerlock rules for aset of potential protected ASes, called target ASes . For eachtarget AS, we advertise a /24 prefix from many points-of-presence (
PoPs ) on the control plane. This is the controladvertisement . It is a normal /24 origination in every way,except that our university AS - which we know not to beprotected by any Peerlock rule - is poisoned (i.e., prependedto the advertisement’s AS PATH - see Section II-D). We thenlisten at varied collection sites, called collectors , for BGPupdates triggered by our advertisement. The AS PATH for eachsuch update that arrives at collectors lists in encounter orderthe ASes that received and re-issued the update as describedin Section II-A.5aken together, the gathered AS PATHs form a directedacyclic graph (DAG) that describes the control advertisement’spropagation through the control plane; each AS appearingon at least one AS PATH forms a node in the DAG, andAS ordering within paths allows us to form directed edgesbetween nodes. BGP loop detection prevents cycles as notedin Section II-D. We call this graph the control DAG . Note thatall of the ASes appearing in the control DAG propagated (andthus did not filter) control updates that include a poisoned AS.We then wait 30 minutes for update propagation beforeissuing an explicit withdrawal for the /24 prefix. This timingis built conservatively from empirical measurements of propa-gation times through the control plane (see update propagationexperiments in appendix). After another waiting period toensure the withdrawal has completely propagated, we issue a leak advertisement for the same /24 prefix. This advertisementmatches the control advertisement in every way, except thatthe target AS is poisoned. This leak advertisement structure isdesigned to mimic a leak for the purposes of Peerlock whileavoiding other common filtering systems. The target AS’spresence on update paths triggers filtering for any Peerlockprotector ASes.Finally, we gather all BGP updates for the leak advertise-ment from our collectors. The ASes that appear on AS PATHsin any of these updates are added to a set called the leak set .Since they propagated poisoned updates, we know these ASesdid not filter the "leak". With the control DAG and leak settogether, we can reason about which ASes are Peerlocking forthe target AS using two techniques: 1) clique inference and 2)DAG inference.Fig. 5: Measurement experiment depiction. Inferences aremade about Peerlock deployment based on differences be-tween normal updates (left) vs. poisoned updates (right) ar-riving at collectors.For detecting Tier 1 protector ASes, we use clique inference .This simple technique relies on the fact that Tier 1 ASesform a peering clique by definition. According to the valley-free routing model [7], ASes share all updates received fromcustomers with their peers; this maximizes the traffic the AStransits for its customers (and thus the AS operator’s compen-sation). Further, ASes should not share a peer’s updates withanother peer, as this is a Type 2 route leak [44]. So, in general,if a Tier 1 AS is observed propagating an update, all
Tier1s should receive the update via their peering relationships.Because we observe at least one Tier 1 propagating controland leak updates across all experiments, we define a simple rule for inferring Tier 1 protector ASes: any Tier 1 AS thatappears in the control DAG but not the leak set is Peerlockingfor the target AS.Inferring other protector ASes requires a more general tech-nique. Outside the structural guarantees provided by the Tier1 clique, there is significantly more uncertainty about whichnetworks are filtering leak updates. Specifically, it is difficultto distinguish an AS filtering updates from an AS not receivingupdates at all due to filtering by other upstream/downstreamnetworks. This challenge leads us to make three separateinferences for these ASes for each leak target.First and simplest is the max inference set, defined as allcontrol DAG ASes minus the leak set. This set includes allASes who may have filtered leak updates, but also ASes whodid not receive the leak update because it was filtered by anintermediate AS. Secondly, we build a min inference set. Thisset is built by deleting all leak set ASes from the controlDAG, and collecting the root of every weakly connectedcomponent that remains. This isolates the ASes that filteredleak updates from ASes in their "shadow" who did not receivethe updates. The min inference set contains those ASes wholikely filtered leak updates based on routes we observed. Notethat the min/max inference techniques closely align with thoseemployed in the long path filtering experiments in Smith etal.’s study on BGP poisoning as a re-routing primitive [40].Our last inference set is the likely inference . Because ASesonly export their best path to our /24 prefix, we cannotobserve every edge that should exist in the control DAG (i.e.,every potential propagation path for updates). So, this set’sis built like the min inference set, except that we augmentthe control DAG with edges from CAIDA’s provider-peerobserved customer cone inference [50]. That is, we add edgesto the control DAG where CAIDA’s data indicates there arelinks between ASes that we did not observe due to policydecisions. This forms a superset of the min inference set anda subset of the max inference set that contains the most likelyfilterers. This is a novel technique not used to our knowledgein any prior work on this topic.These three inference sets are formed for each targetfrom differences in control and leak update propagation. Inaddition to these sets, we also build a min/max/likely poisonfiltering set by following the same steps listed above, butwith a unpoisoned advertisement ’s updates compared againstthe control advertisement’s updates. These sets are built toexplore the prevalence of general poison filtering as in Smithet al. [40].
Framework Details:
The control-plane measurementframework for these experiments consists of 1) 13 PoPs toissue BGP advertisements and 2) 54 BGP collectors to listenfor propagation. We employ the PEERING testbed [35] forthe first requirement. PEERING allows us to advertise threeassigned /24 prefixes from edge routers at thirteen PoPsworldwide. For collecting BGP updates, we used CAIDA’sBGPStream [50] tool. This tool draws updates from 54globally distributed collectors, including 30 RouteViews [31]6nd 24 RIPE RIS [28] collectors. While most of thesecollectors are positioned in North America and Europe, everypopulated continent is represented by at least one collector.
Measurement Limitations:
While our framework allows usto effectively probe the control plane for evidence of Peer-lock and related techniques, a number of limitations preventcomplete certainty regarding Peerlock filter placement. Themost important of these obstacles are imperfect collectorcoverage, topological instability, and the presence of otherfiltering systems. Here we discuss each of these factors inturn.BGP policies prohibit us from viewing the entirety of thetopology with our framework; there are few collectors instub networks, and stub/remote ASes do not export receivedupdates back "up" through provider networks. This means our observation window - the ASes on update paths at collectors- is biased toward transit networks in the Internet’s coreas in [32]. Fortunately, this is the most important/influentialregion to monitor, as these network’s policies have the widestimpact on the control plane. Altogether, we observed 610 ASesduring our experiments, including 181/605 large ISPs and all19 Tier 1 networks. Most observed ASes (332) were presentin the observation window during all experiments conductedfrom August 2019-May 2020. Note that while we can onlyinfer protector
ASes from our observation window, we canpoison any AS. So, our window does not limit our inferenceregarding which ASes are protected .To account for instability in our observation window, welimit our filtering inferences to those ASes observed in controlupdates both before and after the leak advertisement (i.e., forthe current and next target AS experiment). Additionally, werepeat experiments - issue control/leak advertisements for thesame target ASes - over several months. These observationsare combined to reduce the "noise" of topological dynamismfrom our inferences. Specifically, we remove ASes froma target’s filtering inference sets if we later observe thempropagating a leak update for that target; in this case, theearlier inference was likely caused by the AS’s intermittentpresence in the observation window during the experiment.Most importantly, we acknowledge that we cannot be certainPeerlock/Peerlock-lite exactly as described by NTT [43], [18]is responsible for all observed filtering, but our experiments aredesigned to avoid common leak filtering systems. First, sincethe leak and control advertisements in our experiments sharean origin AS/prefix, their updates present identically for ROVfiltering purposes. Additionally, since we observe all ASes inthe control DAG propagating control updates, we infer thoseASes will not apply common IRR or max-prefix limit filteringto the same /24 in leak updates. Finally, while prior workindicates that short poisoned paths are frequently present onthe control plane [48] and rarely filtered [40], the poisoningin the control advertisement ensures that we do not conflatepoison filtering and Peerlocking.Despite our efforts to avoid common filtering practices,local policies grant network operators extensive discretion in how routes are vetted and exported. This flexibility means wecannot be certain that experimental updates are not sometimesblocked by AS specific, ad-hoc AS PATH filtering techniques.We know of no way to distinguish such functionally similarfilters from Peerlock.
Ethics:
We issued only well-formed BGP advertisementsusing the PEERING software client and adhered to all rulespublished by PEERING. We advertised only our assigned /24prefixes, which are reserved for experimental use, and thus didnot disturb Internet control or data plane operation for anynon-experimental IP addresses. Our experiments did requirepoisoned advertisements, but this is a common practice usedboth in research [2], [40] and in traffic engineering [48]. Onenetwork operator observed and inquired about our experimentsto PEERING, but did not report any resultant adverse effects.No data-plane traffic was sent during the conduct of ourexperiments.
B. Evaluation
Target Set 1, Tier 1s : The 19 Tier 1 ASes form our firsttarget AS set, i.e. the potential protected ASes for which weare inferring Peerlock rules. The Tier 1 peering clique includesthe most influential networks by one of the few observablemetrics, customer cone size [50], and often creates [27] ordistributes [23], [11], [39] leaks that disrupt global Internetservices. Paradoxically, deploying filters for leaks that includeTier 1 ASes is also relatively simple for non-Tier 1 networksvia the Peerlock-lite system described above. We iterativelyissued unpoisoned, control, and leak advertisements that cov-ered this set every two months from August 2019 to May2020. This repetition allows us to capture filtering rules forASes with inconsistent presence in our observation window,and to explore how deployments change over time.We first present results for protection within the Tier 1clique in Fig. 6. Note that because of BGP loop detection,every AS filters leak updates that include their own ASNregardless of Peerlock deployment. The peering clique isfortunately the most stable feature in our observation window,enabling us to measure the presence/absence of nearly everypotential Peerlock rule within the clique. We have markedthe exceptions for which we were unable to measure filteringrules in pink in Fig. 6. We see that Peerlock deployment issignificant but unevenly distributed within the clique. SomeASes - e.g. AS 2914 NTT, AS 701 Verizon - filter leakupdates for virtually the entire clique. For five others - e.g.AS 3491 PCCW Global, AS 6762 Telecom Italia - we foundno evidence of Tier 1 Peerlock filtering at all.Our measurement results for Peerlock/Peerlock-liteprotection of Tier 1s by all observed ASes are depictedin Fig. 7. Fig 7a shows both our inferences about whichnetworks filter poisoned updates in general (blue lines) andwhich filter Tier 1 leaks (red lines). These are displayed as acumulative distribution function (CDF) over Tier 1 targets;likely inferred filtering levels range from about 3% (AS6830) to 15% (AS 701) of observed ASes. Note that per7 a) Number of protector/protected rules by ASN. Protector numbersinclude ASes protecting their own ASN via loop detection. (b) Depiction of Tier 1 protection rules.
Fig. 6: Tier 1s filtering Tier 1 leaks, 2019/2020 measurements.the experimental design described above, we cannot makePeerlock protection inferences for ASes filtering all poisonedupdates; however, this is a small set without Tier 1/largeISP members (max inference size = 9 ASes). Fig. 7 showsthe number of ASes in each UCLA class (see Section II)protecting at least one Tier 1 target.
Target Set 2, Tier 1 Peers : Our second target set includesthe non-Tier 1 peers of Tier 1 ASes (about 600 ASes) asinferred by CAIDA [50]. These experiments explore whetherTier 1 ASes are extending Peerlock protection to their non-Tier 1 peers. Additionally, despite covering about 1% of allASes, this set includes a third of all large ISPs. This meanswe can additionally investigate whether non-Tier 1 ASesapply Peerlock-lite filters to large transit networks outside thepeering clique. These experiments were conducted from Oct2019 to May 2020, with every peer ASN targeted at leasttwice.The overall results are presented in Fig. 8a. Clearly, filteringfor these leaks is less prevalent within our observation window.80% of Tier 1 peer leaks were filtered by fewer than 2%of observed ASes, but a few exceptional targets did triggersignificant filtering behavior. Our poison filtering inferencefor these targets is, as expected, nearly identical to thatderived from the Tier 1 leak experiments. Fig. 8b displaysfiltering levels for each Tier 1 ASes by peering status withthe target. All Tier 1s protect 10 or fewer peer networks fromthis set. More variance exists in non-peer filtering behavior;this dynamic will be explored more fully in the followingdiscussion.
C. Discussion
Consistent with Smith et al. [40], we find no evidencefor widespread filtering of otherwise unremarkable poisonedpaths. Their study also found that poisoning high degree ASes in an update is associated with reduced propagation.Specifically, sub-20% update propagation rates were observedfor some Tier 1 ASes, including AS 174 (Cogent/Tier 1)and AS 3356 (Level 3/Tier 1). Birge-Lee et al. [2] likewisefound that using AS poisoning rather than communities asa path export control primitive significantly reduced updatespread, especially when large transit providers were poisoned.Defensive AS-path filtering (e.g.,Peerlock/Peerlock-lite)is identified as a likely culprit for this effect. Our worksystematically examines how and where these filters aredeployed on the control plane (within the limits of ourobservation window).
Tier 1 Leak Filtering : The greatest protection within ourobservation window is clearly afforded to Tier 1 ASes. Ourinitial experiments in August 2019 discovered evidence for133/342 ( − ) possible Tier 1-Tier 1 filtering rules (about39%). Each measurement that followed uncovered at leasttwo new filtering rules, and by our final experiment in May2020, 153 rules had been observed, a nearly 15% uptick inPeerlock deployment. We had previously observed a negativefiltering result for every additional rule, indicating this increaseresults from genuinely new Peerlock deployments rather thaninstability in the observation window.Non-Tier 1 ASes also filter Tier 1 leaks, though thisbehavior is far from uniform. Overall, Tier 1 leak filteringranged from 3% to 15% of observed ASes across Tier 1 AStargets. Most of this is likely due to Peerlock-lite filtering, asit is simpler to deploy. Moreover, fewer than 10% of the morethan 1,000 observed Tier 1 filtering rules exist between peers,and only about 20% (236 rules) involved a Tier 1’s indirectcustomers filtering leak updates. This suggests that ASes areinstalling Peerlock-lite filters for all Tier 1s rather than simplyprotecting their upstream providers.Mutually Agreed Norms for Routing Security (MANRS) [6]8 a) Blue lines show poison filtering; red lines depict Tier 1 leak filtering. (b) Blue bars show no. ASes in observation window; red bars show no.ASes filtering at least 1 Tier 1 leak. Fig. 7: Overall filtering of Tier 1 leaks, 2019/2020 measurements. (a) Overall filtering levels for Tier 1 peer leaks. Max and likely poisoninferences match for this set. (b) Tier 1 filtering of Tier 1 peer leaks (peers within clique excluded).
Fig. 8: Tier 1 peer leaks, 2019/2020 measurements.is an initiative whose ISP members agree to best routing prac-tices (like AS path filtering) to secure inter-domain routing.While Peerlock and Peerlock-lite are not specifically includedin MANRS expected filtering actions, they are both suggestedin the implementation guide [5]. Fig.9 displays as a CDF theproportion of MANRS and non-MANRS ASes filtering Tier1 leaks. 73 of 502 MANRS ASes fall within our observationwindow; the proportion of observed MANRS ASes that filteredTier 1 leaks ranged from 2-18% depending on Tier 1 target.Non-MANRS filtering for the same targets ranged from 2-12%.As shown in Fig 7b, the proportion of ASes with Tier1 leak filters rises with UCLA class [32]. Intuitively,networks with larger customer cones have the resources for sophisticated configurations and the imperative to preventissues for downstream customers, and have previously beenassociated with differing responses to BGP events [40], [3].This dynamic hampers systems requiring wide participationlike ROV [8] and IRR filtering [22], but does not limitPeerlock or Peerlock-lite deployment.
Tier 1 Peer Leak Filtering : Our non-Tier 1 leak experi-ments met with relatively sporadic filtering. For more than80% of targets in this set, nearly every observation windowAS (>=98%) propagated leaks. As described in Section III,Peerlock-lite filters for non-Tier 1 ASes require more carefuldeployment. The outliers in this target set (see the long tailin Fig. 8a) are invariably near-Tier 1 networks like AS 12739odafone, AS 6939 Hurricane Electric, and AS 7843 Charterthat are safe for most ASes to include in a Peerlock-lite filter.Tier 1 filtering of this leak set was likewise reduced com-pared to Tier 1 leaks. In general, Tier 1 networks deployfewer than 5 Peerlock filters for non-clique peers. Nearlyall of these cover near-Tier 1s like AS 7922 Comcast andAS 1273 Vodafone, or ASes administered by Tier 1s e.g.AS 702/703 Verizon and AS 3549 Level 3. Notably, threenetworks extend protection to more than 15 non-peers (perCAIDA’s inference). AS 2914 NTT’s non-peer filtering rulesall cover various Comcast ASNs, while AS 12956 Telefonica’srules appear to be regionally-based: zero rules are applied tocustomer cone ASes, but 23/31 apply to other European ISPsof varying size. 13/20 of AS 3320 Deutsche Telecom’s non-peer filtering rules, on the other hand, cover ASes within itscustomer cone.In summary, Peerlock is widely deployed and expandingwithin the peering clique. Deployment outside the peeringclique is relatively limited, however. Up to 20% of non-cliquenetworks also deploy Peerlock-lite (or a similar mechanism)to filter leaks containing Tier 1 or near-Tier 1 ASes. Thesedeployments are proportionally more common in ISPs andrarely seen in stub ASes within our observation window. Fortu-nately, the effectiveness of Peerlock/Peerlock-lite deploymentsis less sensitive to scattershot deployment than other filteringsolutions. Prior work [8] and our simulations in the followingSection V suggest that filtering by large ISPs can have anoutsize impact on global leak propagation.Fig. 9: Tier 1 leak filtering for MANRS/non-MANRS ASes.V. E
XPLORING P EERLOCK ’ S P RACTICAL I MPACT
The substantial but limited Peerlock/Peerlock-lite filteringmeasured in the previous section leads us to investigate thesesystems’ protective benefit in partial deployment. We haveinterest both in how well these systems protect the controlplane from Tier 1 leaks as deployed, and in the relative effectof realistic additional deployment (e.g. adding filters at largetransit networks). To answer these questions, we quantifythe practical impact of Peerlock with Internet-scale simula-tions that test thousands of leaks against several deploymentschemes.
A. Simulation Methodology
These experiments are conducted via extensions to a BGPsimulator, an approach consistent with prior work on thistopic [37], [36], [41], [48]. We construct a simulated AS-level topology from CAIDA’s inferred relationship dataset(Jan. 2020 data) [50]. ASes within the topology evaluateand export routes using the BGP decision process; longest-prefix matching, LOCAL PREF, and AS PATH guide pathselection, while route export is governed by local policy toenforce valley-free routing. This ensures the simulator modelsthe central dynamic of control plane propagation - the Gao-Rexford model [7], and allows for the closest approximationof control plane behavior we can devise without ASes’ full(private) routing policies.Each simulation is driven by a protection scenario thatmaps protector ASes to those they are protecting. As withPeerlock in practice, these protectors drop all received routesthat transit a protected AS unless they arrive directly from thatAS. Some scenarios also include Peerlock-lite deployments;for these experiments, some set of ASes filter all customer-exported routes that transit Tier 1 ASes. Once we establish theprotection scenario, we iterate over all Tier 1 to Tier 1 links(with 19 Tier 1 ASes, this is n = 19 , n − n = 342 links).These links describe a unidirectional connection from one Tier1, called the link start , to another Tier 1, called the link end .For every link in this set, we sample 20 ASes from the linkstart’s customer cone to serve as leakers . Each leaker will,in turn, randomly select a destination AS in the link end’scustomer cone, and advertise a route to the destination overthe link to all of its peers/providers (see Fig. 10). This modelsa Type 1 route leak of a path over the peering clique [44]. Afterthe leak, we allow the topology to converge and measure howmany ASes 1) received leak updates and 2) installed the leakpath. Additionally, we capture all the AS PATH of all leakupdates for analysis. With 20 leaker/destination pairings perlink and 342 Tier 1 links, our results include 6,840 simulatedleaks.Fig. 10: Example simulated leak. Dashed red lines indicateroute leak to other providers/peers.Our simulations focus on leaks with Tier 1 leaks fortwo reasons. First, we do not find substantial real-worldPeerlock/Peerlock-lite protection of non-Tier 1 ASes as out-lined in Section IV. Second, many consequential leaks arepropagated globally over the Tier 1 backbone, e.g. [27], [23],1039], [11]. Some of our protection schemes will investigatewhether leaks can propagate throughout the Internet withoutTier 1 distribution.
B. Evaluation
We evaluate seven different protection schemes for Tier 1leaks. • No filters . • Inferred : Tier 1 Peerlock levels observed during Internetmeasurements. • Full T1 : All Tier 1s Peerlock for all other Tier 1s. • Full T1 + large ISP lock : Same as full T1, but alllarge ISPs (376 ASes in CAIDA Jan 2020 dataset [50])Peerlock their Tier 1 peers. • Full T1 + large ISP lite : Same as full T1, but all largeISPs deploy Peerlock-lite to protect clique ASes. • Full T1 + large ISP both : Same as full T1, but all largeISPs deploy Peerlock-lite filters and Peerlock for theirTier 1 peers. • Inferred + large ISP lite : Same as inferred, but all largeISPs deploy Peerlock-lite.While it is simpler to filter customer-learned routes withPeerlock-lite than to deploy Tier 1 Peerlock filters for largeISPs, we include both Peerlock and Peerlock-lite filtering bythese ASes to study how leaks are propagated within thetopology. The results of these experiments are presented inFig. 11, which displays both the proportion of ASes in thetopology receiving leak updates (Fig. 11a), and the proportionselecting/exporting the leak path (Fig. 11b).A critical feature revealed by Fig. 11 is the insufficiency ofTier 1 protection alone (blue lines). Full Tier 1 Peerlockingprevents all distribution of studied leaks over the peeringclique, but leak updates still spread to the majority of thetopology for most experiments. Adding large ISP Peerlockprotection has a relatively significant impact on both propaga-tion and installation.Peerlock-lite deployment by these ASes (red lines) benefitsfrom more filterers with wider protection per filterer. Naturally,these scenarios are much more effective at preventing prop-agation. For most leak cases, less than 10% of the topologyreceives leak updates. This highlights the leverage large ISPshave within the topology; filtering at these ASes (<1% of allnetworks) generates an extensive shielding effect. The distinct"shoulder" on the Peerlock-lite curves in Fig. 11b suggeststhe impact on ASes using the leak is even more pronounced.There is virtually no impact on target link usage for 75% ofsimulated leaks when Peerlock-lite is deployed by all largeISPs. Interestingly, the combination of Peerlock and Peerlock-lite filtering by large ISPs (green line) adds little value overPeerlock-lite alone.
C. Discussion
Path Encoding:
To analyze how each of these scenariosshapes leak propagation (and route selection/export), we col-lect the AS PATH of all leaks exported during the aboveexperiments. We use a novel path encoding whereby each
Common Leak Segment EncodingsScenario/Encoding No. exporting ASes % of exporting ASesNo filters
Inferred
Full T1
Full T1 + large ISP lock
Inferred + large ISP lite
Full T1 + large ISP lite
Full T1 + large ISP both
Table 2: Most common encodings with number and percentageof ASes exporting leaks.AS on leak AS PATHs is converted to a 2-tuple with theform (relationship to next AS, UCLA class [32]). Only the ASPATH segment from the first customer to provider link to theleaker ASN - the leak segment - is encoded. This trimmingdiscards the "down" segment prepended as leaks propagatewithin customer cones, as well as the the segment connectingleaker and destination that is invariant across leaks. We includeAS relationship in the encoding because of its importance inpath export behavior as described in Section II-A; UCLA classinforms us regarding where leaks travel through the routinghierarchy. Taken together, these factors help us understandbroadly the topological dynamics at play in leak propagation,and to capture the dominant leak propagation vectors undereach protection scenario.Relationship is encoded as "C" (customer), "R" (peer), or"P" (provider). UCLA classes are indicated by "T" (Tier 1),"L" (large ISP), "S" (small ISP), and "U" (stub). Example:[LR, TP] encodes a leak path exported to a Tier 1 provider bythe leaker, who then passes the leak to a large ISP peer. Theprogress of the leak through the large ISP’s customer conewould continue to the left of "LR", and the path from leakerto destination would continue to the right of "TP", but thesesegments are omitted as explained above.We will use two tables in analyzing our results. Table 1 de-picts the three most common leak encodings for each scenario;these account for at least a quarter of leak paths regardlessof filter placement. We also list the sum and percentage ofASes exporting leaks accounted for by each encoding. Table2 gives summary statistics for leak segments, including their11 a) Impact of various deployment scenarios on leak update propagation. (b) Note increased Peerlock-lite performance for path switching vs. leakupdate propagation.
Fig. 11: Peerlock/Peerlock-lite simulation results.
Transited AS StatisticsScenario Segment Length Tier 1s Large ISPs Small ISPsaverage std. dev % paths average std. dev % paths average std. dev % paths average std. devNo filters
Inferred
Full T1
Full T1 + large ISP lock
Inferred + large ISP lite
Full T1 + large ISP lite
Full T1 + large ISP both
Table 3: Analyzing leak segments by UCLA classes transited.average length and the percentage of leak segments transitingeach UCLA class. Because we do not encode customer conepropagation in leak segments, stubs are transited in <10% ofpaths across all protection scenarios, are are omitted from thetable.First, we observe that even under the "no filters" scenario,leaks re-transiting the Tier 1 clique are not the most commonpath encoding in Table 1. Table 2 shows they are presentin <35% of leak segments under all scenarios. This result isan artifact of the BGP decision process; paths learned fromcustomers are preferred over those exported by peers, and peerroutes are favored over provider-learned ones. So, with allother selection criteria equal, routes exported from providers"above" an AS in the topology - e.g. the peering clique -will generally only be installed and exported if the AS hasnot received an update from peers/customers "below". SinceTier 1 providers cap the routing hierarchy, we expect ASeswill prefer non-Tier 1 routes when provided alternatives bytheir connectivity. This dynamic explains why the additionalprotection afforded by complete Peerlock within the peeringclique vs. current levels is muted in Fig. 11b.This effect also brings large ISPs to the fore in our simula-tions. As noted in [32], these networks are densely connectedwith peering links. Their connectivity allows them to bypassthe Tier 1 clique for many routes - and makes them the primary channel for leak propagation. The most common encodingfor every scenario in Table 1 includes a large ISP, and 18/21of the top encodings transit at least one. More than 70% ofleak segments transit these ASes for all protection scenarios(see Table 2). In fact, in the scenarios without Peerlock-lite(top four listed), leak segments on average transit - and couldbe filtered by - multiple large ISPs. These statistics motivatethe scenarios that place Peerlock-lite filtering at these ASes(bottom three in tables).Interestingly, Peerlock-lite diminishes leak usage and prop-agation unequally as shown in Fig. 11. Fig. 11a shows about20% of leak segments propagate to 20% or more of thetopology with large ISP Peerlock-lite deployment, but Fig. 11bshows that fewer than 5% are installed/exported by at least20% of ASes. Table 1 hints at why this is the case - athird or more of leak segments in Peerlock-lite scenarios areexported to large ISP peers, who propagate them directly intotheir customer cones (indicated by [LR]). Large ISPs withany customer-learned or preferential (e.g. shorter) peer-learnedpaths to the leak destination will prefer their existing route,so the [LR] only includes a subset of the leaker’s peers.Large ISP peers advertising the leak to customers could reachmany ASes, but as a provider-learned route, the leak will bedisadvantaged in the BGP decision process.We see in Table 1 and Fig. 11 that small ISPs do not12ave the connectivity to propagate leaks globally when thelarge ISP provider channel is blocked by Peerlock-lite. Underall scenarios, most leak segments do not transit a smallISP (though they may be transited during propagation intocustomer cones). This feature suggests a less prominent rolein route exchange for these networks relative to large ISPs.To summarize, we find large ISPs are the most criticalplayers in halting the spread and installation of Tier 1 leaks.These networks are interconnected enough to globally dissem-inate route leaks without the peering clique in many cases.Moreover, adding simple Peerlock-lite filters at these ASes tothe currently deployed Peerlock filters in the peering cliquecauses a 94% reduction in total leak export across 6,840 leaksimulations. Table 1 suggests that peer connections amongISPs are the largest remaining vulnerability for Tier 1 leaksgiven uniform large ISP Peerlock-lite deployment. These chan-nels are out of reach for Peerlock-lite as described, but couldbe mitigated by 1) additional peering relationships/Peerlockrules to protect important leak targets and/or 2) complementaryleak prevention systems like IRR filtering.VI. R
ELATED W ORK
Smith et al.’s 2020 study on the efficacy of poison filteringfor inbound re-routing [40] similarly employed the PEERINGframework to probe the behavior of remote networks. Thatwork encountered some evidence for poison filtering, andnoted that filtering rates increase with poisoned AS degree,but did not seek to describe the underlying filtering mechanismor measure which ASes filter poisons. Similarly, Birge-Lee etal. [2] trialed poisons as a primitive for their novel hijacking at-tack, SICO. The authors encountered filtering when attemptingto poison large transit networks and measured the data-planeimpact of reduced update propagation, but did not examinefiltering position or prevalence.Hlavacek et al. [12] introduced the DISCO system forpreventing BGP hijacking. While not designed to prevent routeleaks, the approach taken by DISCO is ideologically similarto Peerlock - DISCO emphasizes deployability/usability at theexpense of some security guarantees relative to RPKI/ROVfiltering. This line of thinking is informed by a long historyof glacial deployment rates for security features that hardenBGP including BGPSec [46] and the RPKI [8], [30].Previous work that relies on BGP poisoning often assumes1) unpoisoned ASes will forward poisoned updates and 2)poisoned ASes will drop such updates (see Section II-D).Katz-Bassett et al.’s LIFEGUARD fault detection and reme-diation system [19], [20], for instance, employs poisoningto steer ASes around link failures. Smith and Schuchard’sNyx defense [41] depends on rerouting with poisons forDDOS/Link Flooding Attack mitigation. Anwar et al.’s pathdiscovery technique [1] is also driven by BGP poisoning.While we discovered little evidence for general poison fil-tering, the prevalence of Tier 1/large transit network filteringcould present an obstacle to these systems. Specifically, theassumption that unpoisoned networks will propagate poisonsdoes not hold in all cases. VII. C
ONCLUSION
This work probes the current deployment ofPeerlock/Peerlock-lite on the control plane with active Internetmeasurements in Section IV. We find substantial evidencefor deployment of these leak defense systems, especiallyin large transit networks, and measure a rise in Peerlockdeployment within the peering clique during our experiments.While the range of protected networks is still narrow withinour observation window, with most filterers protecting onlyTier 1 ASes, many of the most disruptive recent routeleaks contain these networks. Defensive systems [41], [20],measurement techniques [1], and attacks [2] that may poisonPeerlock-protected networks could inadvertently trigger thesefilters, and must not assume poisons will be propagatedby unpoisoned networks. BGP simulators should likewiseaccount for the presence of Peerlock to faithfully reproducecontrol plane behavior.We also examine how the position and prevalence of fil-tering impacts leak propagation in the AS-level topology inSection V. Notably, we find that large ISPs filtering plays amajor role in global leak dissemination, signaling that Tier1 clique deployment of Peerlock alone is not sufficient toisolate leaks. Strategic placement of filters at these large transitproviders, which account for fewer than 1% of all ASes,completely mitigates 80% of simulated Tier 1 route leaks.The MANRS filtering guide encourages AS PATH filteringby member ISPs, particularly for screening customer adver-tisements, and gives Peerlock/Peerlock-lite as examples. Butthese systems are not explicitly required, unlike IRR filter-ing (see [5] Section 4.1.1.1). Given the many indirect/directcustomers these networks serve, ISPs are best equipped andbest incentivized to deploy effective filters. Moreover, neitherPeerlock nor Peerlock-lite is technically complex or burden-some to configure. Therefore, we argue for broad applicationof these common-sense leak prevention techniques by ISPs asa meaningful step in securing inter-domain routing.
A. Operator Engagement
This study’s preprint was posted to the NANOG and RIPEoperator mailing lists in June and August 2020. While operatorresponse was limited, email correspondence around the studydid yield some helpful insights. One European AS operatorclaimed that at least one (and possibly additional) transitnetworks deployed techniques similar to Peerlock around2007, roughly ten years before NTT’s codification of themethod [18]. Separately, a Tier 1 network operator suggestedthat 1) differences in network automation sophistication couldaccount for observed uneven filtering within the peering cliqueand 2) defensive filtering may have partially mitigated theVerizon/Cloudflare leaks detailed in Section II-B. This ideais supported by Cloudflare’s discussion on the incident [23].Cloudflare identifies some networks that filtered the leaks(including ASes 1299 Telia, 2914 NTT, and 7018 AT&T).Bandwidth graphs presented in that post indicate little or noimpact on Cloudflare-to-AT&T data plane operation, while13loudflare traffic to leak propagator Verizon was drasticallyreduced for hours after the incident.
B. Future Directions
Widespread adoption of Peerlock will likely depend onaddressing scalability issues. Rule configuration currently re-quires non-standard, manual out-of-band communication be-tween protector/protected ASes. Automating this process isa crucial step in extending Peerlock beyond core networks.Communities designating authorized upstreams for routes, asproposed in [16], could take the place of out-of-band com-munication. Alternatively, RPKI registration of direct/indirectcustomers [17] could distribute trusted topological informationrelevant to filtering.A
CKNOWLEDGEMENTS
The authors would like to thank Samuel Jero for hishelpful guidance in the shepherding process. The detailed andthoughtful comments/criticisms from our NDSS reviewers arealso appreciated, especially within the context of the ongoingCOVID-19 pandemic. Finally, we would like to recognize JobSnijders at NTT Communications for presenting the Peerlocksystem. This study was supported by the National ScienceFoundation under Grant No. 1850379.R
EFERENCES[1] R. Anwar, H. Niaz, D. Choffnes, Í. Cunha, P. Gill, and E. Katz-Bassett,“Investigating interdomain routing policies in the wild,” in
Proceedingsof the 2015 Internet Measurement Conference , 2015, pp. 71–77.[2] H. Birge-Lee, L. Wang, J. Rexford, and P. Mittal, “Sico: Surgical inter-ception attacks by manipulating bgp communities,” in
Proceedings ofthe 2019 ACM SIGSAC Conference on Computer and CommunicationsSecurity , 2019, pp. 431–448.[3] R. Bush, O. Maennel, M. Roughan, and S. Uhlig, “Internet optometry:assessing the broken glasses in internet reachability,” in
Proceedings ofthe 9th ACM SIGCOMM conference on Internet measurement , 2009, pp.242–253.[4] J. Choi, J. H. Park, P.-c. Cheng, D. Kim, and L. Zhang, “Understand-ing bgp next-hop diversity,” in . manrs . . manrs . org/.[7] Gao, Lixin, “On inferring autonomous system relationships in theInternet,” IEEE/ACM ToN , 2001.[8] Y. Gilad, A. Cohen, A. Herzberg, M. Schapira, and H. Shulman, “Arewe there yet? on rpki’s deployment and security.” 2017.[9] Y. Gilad, O. Sagga, and S. Goldberg, “Maxlength considered harmfulto the rpki,” in
Proceedings of the 13th International Conference onemerging Networking EXperiments and Technologies , 2017, pp. 101–107.[10] V. Giotsas, M. Luckie, B. Huffaker, and K. Claffy, “Inferring complexas relationships,” in
Proceedings of the 2014 Conference on InternetMeasurement Conference , 2014, pp. 23–30.[11] Goodin, Dan, “Google goes down after major BGP mishap routestraffic through China,” https://arstechnica . com/information-technology/2018/11/major-bgp-mishap-takes-down-google-as-traffic-improperly-travels-to-china/, 2018.[12] T. Hlavacek, I. Cunha, Y. Gilad, A. Herzberg, E. Katz-Bassett,M. Schapira, and H. Shulman, “Disco: Sidestepping rpkiâ ˘A ´Zs deploy-ment barriers,” in NDSS , 2019.[13] G. Huston and G. Michaelson, “Rfc 6483 - route origin validation,”2012.[14] IETF, “IRR RPSL Reference Guide,” https://tools . ietf . . irr . net/docs/list . . ietf . org/archive/id/draft-heitz-idr-route-leak-community-00 . txt,2017.[17] M. S. J. Snijders and M. Aelmans, “ RPKI Autonomous Systems Cones,”https://tools . ietf . org/html/draft-ietf-grow-rpki-as-cones-02, 2020.[18] Job Snijders, “NTT Peer Locking,” http://instituut . net/~job/peerlock_manual . pdf, 2016.[19] E. Katz-Bassett, D. R. Choffnes, Í. Cunha, C. Scott, T. Anderson, andA. Krishnamurthy, “Machiavellian routing: improving internet availabil-ity with bgp poisoning,” in Proceedings of the 10th ACM Workshop onHot Topics in Networks , 2011, pp. 1–6.[20] E. Katz-Bassett, C. Scott, D. R. Choffnes, Í. Cunha, V. Valancius,N. Feamster, H. V. Madhyastha, T. Anderson, and A. Krishnamurthy,“Lifeguard: Practical repair of persistent route failures,”
ACM SIG-COMM Computer Communication Review , vol. 42, no. 4, pp. 395–406,2012.[21] Kephart, Nick, “Finding and Diagnosing BGP route leaks,” https://blog . thousandeyes . com/finding-and-diagnosing-bgp-route-leaks/, 2015.[22] B. Kuerbis and M. Mueller, “Internet routing registries, data governance,and security,” Journal of Cyber Policy , vol. 2, no. 1, pp. 64–81, 2017.[23] Levy, Martin, “The deep-dive into how Verizon and a BGPOptimizer Knocked Large Parts of the Internet Offline Monday,”https://blog . cloudflare . com/the-deep-dive-into-how-verizon-and-a-bgp-optimizer-knocked-large-parts-of-the-internet-offline-monday/, 2019.[24] M. Lepinski and K. Sriram, “RFC 8205 - BGPSec protocol specifica-tion,” IETF , 2013.[25] M. Lepinski and S. Kent, “RFC 6480 - An Infrastructure to SupportSecure Internet Routing,” https://tools . ietf . org/html/rfc6480, 2012.[26] Madory, Doug, “Why Far-Flung Parts of the Internet Broke Today,”https://dyn . com/blog/why-the-internet-broke-today/, 2014.[27] ——, “Widespread impact caused by Level 3 BGP route leak,”https://blogs . oracle . . ripe . . ripe . net/publications/docs/ripe-731, 2019.[30] NIST, “RPKI Deployment Monitor,” https://rpki-monitor . antd . nist . . routeviews . org.[32] Oliveira, Ricardo and Pei, Dan and Willinger, Walter and Zhang,Beichuan and Zhang, Lixia, “The (in) completeness of the observedinternet AS-level structure,” IEEE/ACM ToN , 2010.[33] C. Orsini, A. King, D. Giordano, V. Giotsas, and A. Dainotti, “Bgp-stream: a software framework for live and historical bgp data analysis,”in
IMC , 2016, pp. 429–444.[34] J. Schlamp, R. Holz, Q. Jacquemart, G. Carle, and E. W. Biersack,“Heap: reliable assessment of bgp hijacking attacks,”
IEEE Journal onSelected Areas in Communications , vol. 34, no. 6, pp. 1849–1861, 2016.[35] B. Schlinker, T. Arnold, Í. Cunha, and E. Katz-Bassett, “Peering:Virtualizing bgp at the edge for research,” in
Proceedings of the 15thInternational Conference on Emerging Networking Experiments AndTechnologies , 2019, pp. 51–67.[36] Schuchard, Max and Geddes, John and Thompson, Christopher andHopper, Nicholas, “Routing Around Decoys,” in
ACM CCS , 2012.[37] Schuchard, Max and Mohaisen, Abedelaziz and Foo Kune, Denis andHopper, Nicholas and Kim, Yongdae and Vasserman, Eugene Y, “Losingcontrol of the internet,” in
ACM CCS , 2010.[38] P. Sermpezis, V. Kotronis, P. Gigis, X. Dimitropoulos, D. Cicalese,A. King, and A. Dainotti, “Artemis: Neutralizing bgp hijacking withina minute,”
IEEE/ACM Transactions on Networking , vol. 26, no. 6, pp.2471–2486, 2018.[39] Shapelez, Alex, “This is how you deal with route leaks,” https://habr . com/en/company/qrator/blog/495260/, 2020.[40] J. M. Smith, K. Birkeland, T. McDaniel, and M. Schuchard, “Withdraw-ing the bgp re-routing curtain: Understanding the security impact of bgppoisoning through real-world measurements,” in NDSS , 2020.[41] J. M. Smith and M. Schuchard, “Routing around congestion: Defeatingddos attacks and adverse network conditions via reactive bgp routing,”in . IEEE, 2018,pp. 599–617.[42] Snijders, J., Heasley, J. and Schmidt, M., “Use of bgp large communi-ties,” https://tools . ietf . org/html/rfc8195, 2017.
43] Snijders, Job, “Everyday Practical BGP Filtering,” https://peerlock . net,2016.[44] K. Sriram, D. Montgomery, D. McPherson, E. Osterweil, and B. Dick-son, “Rfc 7908 - problem definition and classification of bgp routeleaks,” https://tools . ietf . org/html/rfc7908.[45] Sriram, K. and Azimov, A., “Methods for detection and mitigationof bgp route leaks,” https://tools . ietf . org/pdf/draft-ietf-grow-route-leak-detection-mitigation-02 . pdf, 2020.[46] Sriram, Kotikalapudi and Montgomery, Douglas C., “Resilient Interdo-main Traffic Exchange: BGP Security and DDoS Mitigation,” NIST ,2019.[47] Strickx, Tom, “How Verizon and a BGP Optimizer Knocked LargeParts of the Internet Offline Today,” https://blog . cloudflare . com/how-verizon-and-a-bgp-optimizer-knocked-large-parts-of-the-internet-offline-today/, 2019.[48] Tran, Muoi and Kang, Min Suk and Hsiao, Hsu-Chun and Chiang,Wei-Hsuan and Tung, Shu-Po and Wang, Yu-Su, “On the Feasibilityof Rerouting-based DDoS Defenses,” in IEEE S&P , 2019.[49] UCSD-CAIDA, “CAIDA AS Ranking,” https://asrank . caida . . caida . org/data/as-relationships/, 2020.[51] M. Wählisch, O. Maennel, and T. C. Schmidt, “Towards detecting bgproute hijacking using the rpki,” ACM SIGCOMM Computer Communi-cation Review , vol. 42, no. 4, pp. 103–104, 2012.[52] H. Yan, R. Oliveira, K. Burnett, D. Matthews, L. Zhang, and D. Massey,“Bgpmon: A real-time, scalable, extensible monitoring system,” in . IEEE, 2009, pp. 212–223.[53] Zmijewski, Earl, “Indonesia Hijacks the World,” https://dyn . com/blog/indonesia-hijacks-world/, 2014. A PPENDIX
Update Propagation:
Before (August 2019) and after (May2020) conducting our control-plane experiments in Section IV,we performed simple tests to measure 1) the time distributionfor BGP update arrivals at RIPE/RouteViews collectors fornormal and poisoned advertisements issued from PEERING,and 2) the time distribution for unique ASes seen on ASPATHs in those updates. The latter is most critical for ourexperiments, as we build our filtering inferences from thepresence/absence of ASes on update AS PATHs.These tests consisted of an explicit /24 withdrawal followedby a one hour waiting period, then a normal /24 advertise-ment. We listened for updates for the /24 at all BGPStreamcollectors, and recorded the arrival times of updates for theadvertised prefix for one hour. We also noted when uniqueASes were first seen on the updates’ AS PATHs. This process(withdraw, update, listen) was repeated five times. We con-ducted the same process with a poisoned /24 advertisement,for a total of 10 advertisements per experiment.The results are shown below, Figs.12 and 13. About 80% ofupdates triggered by a normal or poisoned /24 advertisementthat arrived within an hour were received within 30 minutespost-origination in the August experiment. In May, more than95% of updates fell within this period. More importantly, forevery experiment, all unique ASes seen on update paths overthe hour listening window arrived within the first 25 minutespost-origination. Over 95% of unique ASes were seen within7 minutes post-origination. 15 a) August results. (b) May results.
Fig. 12: Update arrival time CDF. Each of five propagation experiments is illustrated in a different color. (a) August results. (b) May results.(a) August results. (b) May results.