On the Estimation of the Number of Unreachable Peers in the Bitcoin P2P Network by Observation of Peer Announcements
OOn the Estimation of the Number of Unreachable Peers in theBitcoin P2P Network by Observation of Peer Announcements
Matthias Grundmann Hedwig Amberg Hannes HartensteinInstitute of Information Security and Dependability (KASTEL)Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany { matthias.grundmann,hannes.hartenstein } @kit.edu Abstract
Bitcoin is based on a P2P network that is used to propagate transactions and blocksof the blockchain. While the P2P network design intends to hide the topology of the P2Pnetwork to impede adversarial actions against peers, information about the topologyis required to understand the network from a scientific point of view, to build realis-tic models used for simulations, and to use these models to optimize protocols of theP2P networks. Thus, there is a natural tension between the ‘desire’ for unobservabilityon the one hand, and for observability on the other hand. On a middle ground, onewould at least be interested on some statistical features of the Bitcoin network like thenumber of peers that participate in the propagation of transactions and blocks. Thisnumber is composed of the number of reachable peers that accept incoming connectionsand unreachable peers that do not accept incoming connections. Despite not acceptingincoming connections, unreachable peers open several outgoing connections to otherpeers and participate in the propagation of transactions and blocks. While the numberof reachable peers can be measured, it is inherently difficult to determine the numberof unreachable peers, exactly because one cannot connect to them. Thus, the numberof unreachable peers can only be estimated based on some indicators. In this paper, wefirst define our understanding of unreachable peers and then propose the PAL (PassiveAnnouncement Listening) method which gives an estimate of the number of unreachablepeers by observing addr messages that announce active IP addresses in the network.The PAL method allows for detecting unreachable peers that use flags to indicate thatthey provide services useful to the P2P network. In conjunction with previous methods,the PAL method can help to get a better estimate of the number of unreachable peers.We use the PAL method to analyze data from a long-term measurement of the BitcoinP2P network that gives insights into the development of the number of unreachablepeers over more than five years from 2015 to 2020. Results show that about 31,000 un-reachable peers providing useful services were active per day at the end of the year 2020.An empirical validation indicates that the approach finds about 50 % of the unreachablepeers that provide useful services.
In Bitcoin [9], transactions and blocks are propagated by peers that are connected in apeer-to-peer (P2P) network. The protocol used for the propagation of transactions andblocks affects the whole system’s performance security: The propagation delay of blocks isconnected to the number of forks, i.e. temporal inconsistencies in the blockchain [2]. Theprotocol’s bandwidth consumption limits the maximum number of connections [10] whichlimits the resilience against eclipse attacks [7]. The protocol can also leak information aboutthe creator of a transaction [4] and partial information about the topology of the network1 a r X i v : . [ c s . CR ] F e b
11, 6, 3]. Analyzing the protocol with regard to these aspects and validating proposals forimprovements can hardly be done in the live P2P network. Thus, researchers and developersuse simulations of the P2P network for these tasks [12, 10]. Such simulations require a modelof the topology of the P2P network. However, the real topology of the Bitcoin P2P networkis unknown to the public because the protocol is designed to hide information about thetopology that could support adversarial actions against peers [4, 7]. Therefore, simulationsmodel the topology based on certain characteristics that can be measured or estimatedwithout having a method for the measurement built into the protocol itself. One of thesecharacteristics and the object of this paper is the number of peers that participate in thepropagation of transactions and blocks.To form the P2P network, each peer creates by default 10 outgoing connections to otherpeers. A peer is a running instance of a Bitcoin software that is connected to at least oneother instance of a Bitcoin software. Peers are encouraged to connect to multiple peers toreduce chances of being victim of an eclipse attack [7]. Not every peer accepts incomingconnections, e.g., because a peer is behind a NAT or a firewall or a peer is configured toblock incoming connections. Thus, the peers can be categorized into two groups: reachablepeers that accept incoming connections and unreachable peers that do not accept incomingconnections. Despite the name, unreachable peers play an active role in the P2P network.While unreachable peers do not accept incoming connections, they open several outgoingconnections to other peers and participate in the propagation of transactions and blocksjust as reachable peers do. Hence, for the propagation of transactions and blocks both, thereachable and unreachable peers are relevant. Franzoni and Daza [5] recently presented howthe robustness and efficiency of the P2P network can be improved by giving unreachablepeers a special role in the propagation of transactions.There exist projects continuously measuring the number of reachable peers. However,unreachable peers are mostly invisible because one cannot connect to them. Although wecan define the set of unreachable peers, we cannot retrieve it using measurements: We canonly estimate the number of unreachable peers. There are two ways to get such an estimate:The first way is to observe a fraction of unreachable peers and extrapolate the whole numberof unreachable peers. This can be done by running a reachable peer that accepts incomingconnections from unreachable peers. The second way is to observe effects that are causedby unreachable peers and infer their number from these observations. In this paper, wepresent an approach that uses the second way to estimate the number of unreachable peers.This approach, that we call the PAL (Passive Announcement Listening) method, relies onobserving address announcements that are forwarded by reachable peers in the network.The approach is only able to count unreachable peers that offer services to the networksuch as storing the latest 288 blocks of the blockchain. Addresses of unreachable peers thatdo not announce such useful services will not be forwarded by other peers and thus not becounted. As there is no ground truth available, we validate our approach by verifying ourassumptions and by comparing the results of our approach to an observation of a fraction ofunreachable peers. We show that the approach detects most reachable peers and about 50 %of unreachable peers that provide useful services. Previous work has estimated the numberof unreachable peers to be around 16,000 peers [11], 54,000 peers [10], 100,000 peers [1], and155,000 peers [15]. The wide range of differences comes not only from different measuringtimes and methods but also from the fact that different definitions of the term unreachablepeer are used. Approaches to gain (partial) information about the topology have been proposed [1, 8, 11, 6, 3]. However,they either do not work anymore or require a strong attacker to get the complete P2P network’s topology. E.g., https://bitnodes.io/ and https://dsn.kastel.kit.edu/bitcoin/ unreachable peers and other relevant terms in thefollowing section. An overview of related work will be given in Section 3. We describe theBitcoin protocol and the behavior of the most common Bitcoin implementation in Section 4.Then, we present the PAL method in Section 5 and present the results of applying themethod to data collected from the Bitcoin P2P network between 2015 and 2020. We validatethe method in Section 6 and conclude in Section 7.
We refer to an implementation of the Bitcoin protocol as Bitcoin software. As stated above,we define a peer as a running instance of a Bitcoin software that is connected to at leastone other running instance of a Bitcoin software. We expect most peers, however, to beconnected to multiple peers in order to reduce chances of being a victim of an eclipseattack. A Bitcoin P2P network consists of peers that are directly or indirectly connectedto each other. Because this definition allows multiple Bitcoin P2P networks, we refer to the
Bitcoin P2P network as the Bitcoin P2P network that includes the peers that mine blockswith more computation power than the peers in every other Bitcoin P2P network. In thefollowing, we consider only peers that are part of the
Bitcoin P2P network.Categorizing peers into reachable and unreachable peers is more difficult than it mightseem at a first glance. A first approach would be to define a peer as unreachable if allother peers cannot initiate a connection to that peer. A peer might be unreachable becauseit runs behind a firewall that blocks incoming connections. However, a peer might acceptincoming connections from one group of peers but refuse incoming connections from otherpeers. Imagine a private network that blocks incoming connections from the outside to peersinside the network but allows for incoming connections between peers that are inside theprivate network (see Fig. 1a). Because a peer in such a network would accept incomingconnections from some peers, this peer would not be called unreachable following the abovedefinition of unreachable. However, this peer would be unreachable for all peers outside theprivate network and, thus, should by intuition be categorized as unreachable. Therefore,a definition of unreachable peers may not be too strict. We consider such cases in thefollowing definition that we use in this paper: A peer p is called unreachable if the majorityof other peers cannot initiate a connection to p . This means that a peer is called reachableif most other peers can initiate a connection to that peer. Note that, if the majority ofpeers were inside a private network that blocks connections from the outside but allows forinternal connections, this definition would categorize the peers in this private network asreachable (see Fig. 1b) because the majority of other peers is in the same private network.Indeed, the peers in this private network might have many incoming connections and, thus,look like reachable peers. However, as peers outside the private network cannot initiate aconnection to them, the peers in the private network would seem unreachable to the rest ofthe world. We think that a binary classification into reachable and unreachable peers wouldnot be suitable for such a scenario. For this work, however, we assume that the Bitcoin P2Pnetwork follows mostly the model as sketched in Fig. 1a, i.e. most peers in the Bitcoin P2Pnetwork accept incoming connections from either all other peers or none.Our goal is to estimate the number of unreachable peers. Because peers join and leavethe network ( churn ) this number changes continuously. The number of peers at a givenpoint in time can be different from the number of peers that existed during a given timeperiod (e.g., during one day). In the following, we will talk about estimating the number ofunreachable peers during time periods. Using a model for churn, this number can be usedto estimate how many unreachable peers existed at a given point in time.3 eachable peers Unreachable PeersPrivate network Reachable peers
Unreachable PeersPrivate network a) b)
Legend
AA B
Set of peers A. Size corresponds to cardinality.Each peer in A can initiate a connection to each peer in B.
Figure 1: Two simple models of how the peers in the Bitcoin P2P network might be con-nected. a) Each unreachable peer can initiate a connection to each reachable peer. A reach-able peer can initiate a connection to another reachable peer. Additionally, there mightbe a private network of unreachable peers that can connect to each other. b) The numberof peers in the private network might exceed the number of other peers. In this case, ourdefinition would classify the peers in the private network as reachable peers although theyare not reachable by peers outside the private network. This is not a problem for this workbecause we assume that the Bitcoin P2P network looks similar to the model a).The peers in the Bitcoin P2P network are identified by their addresses. A peer can havemultiple addresses (in the most common case an IPv4 address and an IPv6 address) andmultiple peers can share an address (e.g., an IPv4 address because they are behind the sameNAT). In this work, we will make the simplifying assumption that each peer has exactlyone address. If we simply use the term address, then it refers to any type of address beingused in the Bitcoin protocol, e.g., IPv4 address, IPv6 address, or Tor address (see BIP 155 for full list). The number of reachable peers is continuously measured by different projects (see Footnote 2on page 2). They share the basic approach of recursively searching the network for peers.Exemplary, we explain the approach of Bitnodes which is similar to that of [13]: Thesoftware starts with an initial set of peers, connects to each peer and requests addressesfrom this peer using a getaddr message. On receiving an addr message as reply, thesoftware tries to connect to each of the addresses in the reply and, for each successfullyopened connection, addresses are requested over this new connection. The set of peers thata connection has been established to, is regarded as the set of reachable peers. In case aconnection to an address cannot be established, it is unknown whether there is no peer atthis address or there is an unreachable peer at this address. Therefore, this approach is notcapable of measuring the number of unreachable peers.In previous work, only few attempts have been made to estimate the number of unreach-able peers. In May 2017, Wang and Pustogarov [15] ran 102 reachable peers as probes in https://github.com/bitcoin/bips/blob/master/bip-0155.mediawiki https://github.com/ayeowch/bitnodes/ version protocol version, services, user agent, address of receiving peer, . . . verack – getaddr – addr number of entries, for each entry: timestamp, services, address, portdifferent data centers around the globe for seven days and logged all incoming connectionsand associated information. For each peer that connected to one of the probes, they triedto open a TCP connection on port 8333 to that peer’s IP address and to open a connectionvia the Bitcoin protocol. They observed on average about 10,000 unique IP addresses in a6-hours interval. They assume an average of 3.5 connections per unreachable peer and 5,540reachable peers and use these numbers to estimate that there were at least 155,000 un-reachable peers in each 6-hours interval. The authors measured that 34.6 % of the observedconnections lasted shorter than one second and 93.9 % lasted shorter than one minute.To parameterize their simulation, Naumenko et al. [10] used numbers obtained froma website run by Luke-Jr . They obtained the information that the network had 54,000unreachable peers (called non-listening) and 6,000 reachable peers (called listening). At thetime of writing (January 2021), the website lists about 50,000 unreachable peers and 5,000reachable peers. The methodology behind the website is not publicly documented, but, inthe absence of other reference points, we also compare our measurements to the numbersobtained from this website. The protocol for peers in the Bitcoin P2P network is described by a public developerreference . The protocol does not distinguish between reachable and unreachable peersbecause a peer cannot detect whether it is unreachable or is reachable but does not haveany incoming connections, yet. Because most of the peers run the same software, BitcoinCore (announced in version messages as ”Satoshi”), the behavior of this software is thede facto specification of the protocol. In the following, we start by describing the protocolrules that all peers should follow and then describe the relevant behavior of Bitcoin Corein more detail. Table 1 gives an overview of the messages in the Bitcoin protocol relevant for this paper. Webriefly explain these messages in this paragraph. Peers need to know the addresses of otherpeers to be able to connect to them. To this end, addresses are propagated in the BitcoinP2P network using addr messages. An addr message consists of a header, the number ofentries that follow, and one or multiple entries. Each entry consists of an address, a port,a timestamp, and service flags. The timestamp was originally meant to describe when thatpeer was seen last, however, the behavior of the reference software was modified to not https://luke.dashjr.org/programs/bitcoin/files/charts/historical.html https://developer.bitcoin.org/reference/index.html https://dsn.kastel.kit.edu/bitcoin/ addr messages to contain up to 1000 entries. addr messages can be sent unsolicited and theycan be requested using a getaddr message. A peer replies to a getaddr message with an addr message.When a connection between two peers is established, they send version messages toeach other that contain the peers’ user agents and services. The receiving peer replies to a version message by sending a verack message. By default, Bitcoin Core opens eight outgoing connections for full-relay and two outgoingconnections that are used only for relaying blocks but not for transactions and addresses.Thus, an unreachable peer maintains outgoing connections to ten other peers. A reachablepeer can additionally have incoming connections. The default maximum number of all (out-going and incoming) connections is 125. In case a new incoming connection fills the lastavailable slot, Bitcoin Core will evict an existing connection based on different metrics suchas duration of the connection, ping times, and amount of transmitted data.Peers request addresses from other peers using the getaddr message which is answeredby an addr message. The reply contains at most 23 % of the addresses in the replying peersdatabase and at most 1000 addresses. addr messages are also sent unsolicitedly: A peerannounces its address to a connected peer once a connection has been established. Eachpeer also regularly announces its address to its connected peers (except those for block-relayonly). The announcements are sent at random times following an exponential distributionwith a mean of 24 hours. In contrast to replies to a getaddr message, these announcementsare forwarded so that peers in the network learn about other peers. To distinguish betweenforwarded announcements and an announcement of a peer’s own address, we call the latera self announcement .To tell whether an addr message was received unsolicitedly or in reply to a getaddr message, Bitcoin Core checks the number of entries in an addr message. Because an-nouncements of addresses are unsolicited, most of the messages with announcements aresmall messages with ten or less entries. Thus, Bitcoin Core considers the addresses in an addr message for propagation if the addr message contains ten or less entries. An addressis only propagated if it meets the following requirements: (1) The service flags associatedwith the address need to have the node witness flag set and the node network or node network limited flags. (2) The timestamp associated with the address must notbe older than ten minutes. (3) The address itself must be routable, i.e., it may not be froman IP address range that is reserved for private use. If the decision is for an address to bepropagated, then it is sent to one or two connected peers. In this section, we present the PAL method and give details on the setup and data collectionand the methodology for analyzing the data. We discuss the limitations of this approachand present the results of applying this method to data collected during a timespan of fiveyears.For the PAL method, we run a passive monitor node that is connected to all reachablepeers. The monitor collects unsolicitedly sent addr messages that are received from itspeers. The set of all addresses received throughout a day results in an estimate of the active6 onitor
Reachable Peers
Unreachable Peers
Figure 2: Overview of the setup. The monitor node is connected to all reachable peers.Unreachable peers are connected to reachable peers, too. There are connections betweenreachable peers but no connections between unreachable peers.peers during this day. After removing the addresses of reachable peers from this set, wehave an estimate of the set of unreachable peers.The PAL method can be seen as a refinement on the approach for finding reachablepeers explained at the beginning of Section 3. This approach itself is unsuitable for reliablyfinding unreachable peers because it only reveals that there are reachable peers at addressesat which an incoming connection is accepted. If no incoming connection is accepted at anaddress, the approach does not allow to draw any conclusions on whether there is no peeror an unreachable peer at this address. Instead of relying on getaddr messages to quicklycollect many addresses, we use a passive monitor node that waits for unsolicited addr messages. An address is only sent unsolicitedly if it is forwarded or if the sending peerannounces this address as its own address (self announcement). If we receive an address inan unsolicited addr message at the monitor, we can conclude that at most ten minutesago there was a peer at this address because these messages are only propagated until thetimestamp associated with the address is older than ten minutes. Collecting all unsolicitedlysent addresses during one day gives us an estimate of the set of peers during this day. Byfiltering out reachable peers, we receive an estimate of the set of unreachable peers for thisday.
Data Collection
We run a monitor node that connects to all known reachable peers inthe network (see Fig. 2). The monitor is mostly passive, i.e. it does not send any messageswith the following exceptions: • After opening a connection, the monitor sends its own identifier in a version message • After having received another peer’s version message, the monitor not only sends a verack message but also requests addresses by sending a getaddr message. • Every two minutes, the monitor sends a getaddr message to a uniform randomlychosen peer.Sending the getaddr messages is not required for the PAL method but also does notinterfere with our results; the addr messages that are received in reply to these messageswill be ignored in the following. The monitor tries to connect to each address it receivesin addr messages (rate-limited per address to once every six hours). The monitor logs allreceived addr messages, version messages and the time when a connection to anotherpeer is established or closed. 7able 2: Overview of notation (also see Fig. 3)Symbol Description A t Set of addresses received on day t in small addr messages (senders excluded) P t Set of addresses that the monitor node was connected to on day tU t (Estimation of) Set of addresses of unreachable peers on day t All
ADDR messages received at monitor during interval 𝑡 Small
ADDR messages received during interval 𝑡 Reachable peers during interval 𝑡 Reachable peers during interval 𝑡 − 1
Outbound connection is still active
Reachable peers in small
ADDR messages received during interval 𝑡 ∩ Unreachable peers in small
ADDR messages received during interval 𝑡 \ Incoming connections at probe(s) during interval 𝑡 Incoming connections from reachable peers at probe(s) during interval 𝑡 Incoming connections from unreachable peers at probe(s) during interval 𝑡 Unreachable peers during interval 𝑡 Extrapolation ∩ Incoming connections at probe(s) during interval 𝑡 visible in small ADDR messages
Legend
White background: unreachable peersGrey background: reachable peers | |
Green border: peers with useful services | |
Blue border: peers without useful services
PAL METHODPREVIOUS WORK 𝑀 t 𝐴 𝑡 𝑃 𝑡 𝑈 𝑡 𝑃 𝑡−1 𝑅 𝑡 𝐼 𝑡 Incoming connections at probe(s) during interval 𝑡 𝑆 𝑡 𝐻 𝑡 VA L I D A T I O N Figure 3: Data flow of the PAL method and the approach of [15] to estimate the numberof reachable and unreachable peers in the Bitcoin P2P network. The sets M t and I t arecollected during measurements and the arrows show filters and operations to derive morespecific sets during the analysis. The border color of each box indicates whether the respec-tive set contains peers with useful services only or also those without. The background colorof each box indicates whether the respective set includes reachable and unreachable peers. Data Analysis
We analyze the logs created by the monitor with the goal of learning thenumber of peers in the network. We describe this process in the following and depict it inthe upper part of Fig. 3. The most relevant notation symbols are listed in Table 2. For eachday t , we collect all addresses that were received by the monitor (see M t in Fig. 3). Wedefine the set A t by applying the following two filters on these addresses: (1) We select onlythe addresses in small addr messages (we define small for addr messages as containingten or less entries). (2) We ignore the self announcements of (reachable) peers, i.e. entriesof an addr message that equal the address of the sender of this addr message. The set A t includes addresses of reachable and unreachable peers that were announced on this day. Wedetermine the set P t of all addresses that the monitor node was connected to on day t . Forthis, we collect all addresses that the monitor already was connected to at the beginningof day t or a connection was established and a version message received during day t . Weconsider this set P t as the set of all reachable peers at day t . Our estimate of the unreachablepeers U t for day t is U t = A t \ P t . 8 Unique addresses in small ADDR messages per day A t Unreachable addresses in small ADDR messages per day U t Figure 4: Number of unique addresses observed in small addr messages per day. Note thatthe upper part uses a different scale than the lower part.
Limitations
As described in Section 4, peers running Bitcoin Core only forward addressesthat have specific flags set ( node witness and ( node network or node network -limited )). Thus, we do not expect to receive addresses of peers that do not have these flagsset at the monitor node. The PAL method can only detect such peers with useful serviceswhich is indicated by the border colors in Fig. 3.Another limitation of the PAL method is that it cannot distinguish whether an unreach-able peer existed only for a short moment on a day or the whole day. Also, the addresses andassociated information in addr messages are not authenticated. Thus, a peer can send anyaddress and information about this address to other peers in the network. If a peer sendsaddresses in an addr message, there is no proof that the peer has received the address fromanother peer or is connected to a peer with this address. Therefore, the approach can bedisturbed by flooding the network with bogus addresses. Measurements
We applied the method to data collected from 2015 to 2020 and presentthe results here. Figure 4 shows | A t | , the number of unique addresses received in smallmessages for each day t and the number | U t | of addresses that were unreachable. The plotshows that, at the majority of days in the observation range, between 20,000 and 60,000unique addresses were received in small messages. Most noticeably, the plot shows a highamount of addresses at the end of the year 2018 and beginning of the year 2019 whichwe will discuss later. The remaining plot shows that the number of addresses increasedfrom the beginning of the year 2017 on to a maximum of about 72,000 addresses in themiddle of December 2017. From this point on, the number of addresses declined slowly untilthe prominent peaks around the turn of the year 2018/2019 and since then the numberof addresses has been relatively stable around 37,000. The number of unreachable peers | U t | has a similar development but is consistently about 28 % lower than the number of alladdresses. At the end of the year 2020, the number of unreachable peers | U t | equals about31,000 peers.The peak at the end of the year 2018 seems like many unreachable peers joined thenetwork within a few days. An alternative explanation would be that bogus addresses weredistributed that do not actually belong to peers. We examined the addresses that werereceived only during this time and did not find any irregularities with regard to theirdistribution in the IP address space, autonomous system, or country of autonomous system.9 P t Reachable IP addresses in ADDR messages per day R t Percent of reachable in ADDR messages per day
Figure 5: Number of unique addresses of reachable peers observed in small addr messagesper day.However, for the highest peak in March 2019, we found that this peak was caused by manyIP addresses from the same /8 subnet. As IP addresses from this subnet were only veryrarely observed before and after March 2019, we assume that this effect was caused byunknown actions of one party that flooded the network with these IP addresses. It is anopen question whether a known or unknown attack would cause such effects and whetherthis can be verified using the data from our measurements.
To validate the PAL method, in this section we analyze the measured data and compare itto other sources.
As a first step, we verify that the results of the PAL method are consistent in itself. Ifour assumptions hold true, we should be able to find all reachable peers in small addr messages during each day. As we know the set of reachable peers, we can use it to evaluatethe precision of our measurements. Putting this into the context of Fig. 3, this means that,if the PAL method works perfectly, we expect that set R t equals set P t .In Fig. 5, we plot the number | P t | of reachable peers for each day t and the number | A t ∩ P t | = | R t | of reachable peers whose addresses we received on each day t . The plotshows that since July 01, 2016 on each day on average 91.4 % of the addresses of reachablepeers were received in a small addr message on the same day (excluding events for peerssending their own address). Over the time, the quality has increased: Since Jan 01, 2017 theaverage is 93.4 % and since Jan 01, 2020 the average is 95.3 %. This is a promising resultas it indicates that reachable peers are consistently found by the PAL method. However, ascan be seen in Fig. 3, we do not have an independent ground-truth for the set of reachablepeers. Because all these sets of peers are based on the contents of addr messages, we canonly conclude that the PAL method’s constraint of looking at one day only and at smallmessages only does not reduce the set of detected reachable peers.10 Figure 6: Stacked plot of the number of unique addresses observed in small addr messagesper day at two monitors. The plot is separated into the number of addresses received onlyat monitor 1 (purple), the number of addresses received at both monitors (green), and thenumber of addresses received only at monitor 2 (blue). It can be seen that the majority ofaddresses is received at both monitors.
60 70 80 90 100Jul'15 Jan'16 Jul'16 Jan'17 Jul'17 Jan'18 Jul'18 Jan'19 Jul'19 Jan'20 Jul'20 Jan'21 0 10000 20000 30000 40000 50000 60000 70000 80000Ratio of IPs seen at monitor 1 that were also seen by monitor 2 (in %, left axis)Unique IPs in small messages per day on seen on both monitors (right axis)
Figure 7: Ratio of IPs seen at monitor 1 that were also seen at monitor 2. The green plotshows the number of unique addresses observed in small addr messages per day at bothmonitors (right y-axis). The green plot is the same as the green plot in Fig. 6.
For further validation, we run a second monitor node with the same method as describeabove. We compare the addresses received by two monitor nodes. If the measurementmethod is reproducible, the data of both monitors should largely overlap. Figure 6 andFig. 7 show the results for two monitor nodes for the whole time of measurement. It can beseen that since the beginning of 2017, over 95 % of the addresses observed at monitor 1 arealso observed at monitor 2. This indicates that the measurement is reproducible and thatthe view of our monitor node is not subjective to the specific instance of the monitor.
If our assumptions hold true, we can expect an unreachable peer that runs continuouslyto be visible in small addr messages every day. We run a small experiment to test thishypothesis. We start an unreachable peer p U running the Bitcoin Core software in version110.20.1 with the only modification to create a log entry when the peer announces its ownaddress. The peer p U has an IPv4 and an IPv6 address. The log entry consists of a times-tamp, which address is announced, and which peer the address is announced to. The peer p U runs continuously and it is unreachable because its incoming TCP port 8333 is blockedby a firewall. After being started, p U creates ten outgoing connections (eight for full-relayand two for block-relay only). After the peer has been continuously running for 44 days,we analyze the logs and compare them to our monitoring logs. We first look at the numberof announcements that the peer p U has sent on each day. While one might expect that p U sends on average eight announcements per day because it announces its address to everyconnected full-relay peer on average every 24 hours, the average measured is about 16. Thisis because although the peer p U is running continuously, connections are closed by otherpeers and p U creates new outgoing connections on which p U announces its address. Thenumber of announcements sent on a day is composed of the announcements on newly cre-ated connections and regular announcements on existing connections. This fact increasesour expectations to receive an address of p U at the monitor on every day. Next, we exam-ine this proposition by looking for p U ’s addresses in the monitor’s logs. We find that themonitor received p U ’s address on 43 of the 44 days. Assuming that this observation can begeneralized, this indicates that the monitor sees the addresses of continuously running peersrunning the Bitcoin Core software with high probability on each day and the PAL methodcan detect these peers. We leave validation of this hypothesis for other Bitcoin software forfuture work. The approach of [15] is to run many public peers that accept incoming connections andobserve the connections they receive which are partially from unreachable peers. For furthervalidation of the PAL method, we use a similar approach and run a single public peer p I that accepts incoming connections. We compare the collected data to the data found in addr messages to find out how many unreachable peers seen by p I are also found using thePAL method.The peer p I does not open outgoing connections and logs for each connection when itis established and closed and the received version messages. We analyze the logs after p I has been running for about 16 months. On average, the peer p I had incoming connectionsfrom about 2,040 different addresses per day (this corresponds to the size of I t in Fig. 3).As addr messages contain almost only addresses of peers providing useful services, weonly select such addresses for comparison. The resulting set per day contains on averageabout 946 addresses (this corresponds to | S t | in Fig. 3). The intersection of this set with alladdresses received in small addr messages on the same day, results in the set of addressesthat opened a connection to p I , announced useful services, and were observed in a small addr on the same day (see H t in Fig. 3). If the PAL method worked perfectly, than thisset H t would equal the set S t in Fig. 3. We find that on average 68 % of the addresses ofincoming connections with useful services were also observed in small addr messages onthe same day. As can be seen in Fig. 3, the set H t contains reachable and unreachable peers.To validate the detection of unreachable peers only, we reduce this set to unreachable peersonly by subtracting the set of reachable peers P t . This shows that about 51 % of incomingconnections of unreachable peers are detected by the PAL method in small addr messages.Analogously, we find that the PAL method detects about 98 % of incoming connections ofreachable peers. Before the experiment, a peer existed that had the same address as p I and initiated outgoing connections.Therefore, p I ’s address is contained in other peer’s databases and peers open incoming connections to p I . As there is no ground truth that we could compare the data from the PAL method to,we compare it to measurements created by previous works. In 2014, Biryukov et al. [1]estimated that the Bitcoin P2P network had a size of 100,000 peers of which 90 % wereestimated to be unreachable. This estimation is too early to be compared to our data but,taking the unreachable peers without useful services into account, this estimation could fitin with our observations. Neudecker et al. [11] simulated the Bitcoin P2P network in 2016and estimated from the simulated propagation behavior that the P2P network had about16,000 unreachable peers that participate in the propagation of transactions and blocks.The PAL method calculates about 14,000 unreachable peers per day averaged over the year2016. As the results of [11] are given for one point in time and the PAL method estimates thenumber of unreachable peers during one day, we would rather expect that the PAL methodwould find more unreachable peers than [11]. Our explanation is that the lower number ofunreachable peers detected might again be caused by peers not announcing useful servicesor by the effects mentioned at the end of Section 6.4.A measurement of unreachable peers was conducted by Wang and Pustogarov in 2017[15]. They ran more than 100 reachable peers to get incoming connections. From theirmeasurements, they estimated at least 155,000 unreachable peers to be active in each 6-hourstime period. This estimate is higher than the results obtained through the PAL method.An explanation for this is that Wang and Pustogarov report that 80 % of unreachable peerswere mobile peers that had only short-lived connections. We assume that these peers eitherdid not announce their IP addresses or that they did not provide useful services. In thiscase, they would be invisible to the PAL method which explains the difference to our results.The measurement by Luke-Jr gives an estimate of the number of reachable and un-reachable peers over a similar timespan. In Fig. 8 we plot our data and the data fromLuke-Jr in one plot for comparison. It can be seen that the number of unreachable peersin the data from Luke-Jr is higher compared to the number of addresses in small messageswhich is probably accounted for by the restriction that only addresses of peers providinguseful services are propagated. The increase in addresses in small messages at the end ofthe year 2018 cannot be seen in the data from Luke-Jr which indicates that these peaks arenot caused by many peers joining the network during this time. However, it seems that thisperiod has had an effect on the measurements by Luke-Jr, too, because once the number ofaddresses in small messages had declined in March 2019, the number of unreachable peers asmeasured by Luke-Jr starts to rise. One interpretation of this fact is that the high numberof addr messages at this time, hid unreachable peers from Luke-Jr’s methodology and theybecame visible only after the number of addr messages had stabilized. https://luke.dashjr.org/programs/bitcoin/files/charts/historical.html A t Reachable addresses per day P t Luke-Jr Reachable addressesLuke-Jr Unreachable addresses
Figure 8: Comparison between the number of addresses observed in small addr messagesper day and the data obtained from the website run by Luke-Jr.
We have presented the PAL method to estimate the number of peers in the Bitcoin P2Pnetwork including unreachable peers. The PAL method adds a further reference point toexisting approaches measuring the number of unreachable peers. The estimate of the numberof unreachable peers at the end of 2020 is about 31,000 peers which, as indicated by ourvalidation, might correspond to about 50 % of all peers providing useful services. Looking atthe measurements of the past five years, we have seen that the number of peers varied. Asprevious work considered only shorter time intervals, their measurements might have beenduring a time when the number of peers was higher than today. The insights gained fromthis work can be valuable for development and parameterization of simulators that modelthe Bitcoin P2P network more realistically.To further improve on the PAL method and the validation, we plan to simulate thepropagation of addr messages. We expect that this can enhance our understanding of themethod and of effects that are visible in the measurements.
References [1] Biryukov, A., Khovratovich, D., Pustogarov, I.: Deanonymisation of clients in BitcoinP2P network. arXiv:1405.7418 [cs] (May 2014), http://arxiv.org/abs/1405.7418 ,arXiv: 1405.7418[2] Decker, C., Wattenhofer, R.: Information propagation in the Bit-coin network. In: IEEE P2P 2013 Proceedings. pp. 1–10 (Sep 2013).https://doi.org/10.1109/P2P.2013.6688704, iSSN: 2161-3567[3] Delgado-Segura, S., Bakshi, S., P´erez-Sol`a, C., Litton, J., Pachulski, A., Miller, A.,Bhattacharjee, B.: TxProbe: Discovering Bitcoin’s Network Topology Using OrphanTransactions. In: Goldberg, I., Moore, T. (eds.) Financial Cryptography and Data Se-curity. pp. 550–566. Lecture Notes in Computer Science, Springer International Pub-lishing, Cham (2019). https://doi.org/10.1007/978-3-030-32101-7 32[4] Fanti, G., Viswanath, P.: Deanonymization in the Bitcoin P2P network. In: Proceedingsof the 31st International Conference on Neural Information Processing Systems. pp.1364–1373. NIPS’17, Curran Associates Inc., Red Hook, NY, USA (Dec 2017)145] Franzoni, F., Daza, V.: Improving Bitcoin Transaction Propagation by LeveragingUnreachable Nodes. arXiv:2010.15070 [cs] (Oct 2020), http://arxiv.org/abs/2010.15070 , arXiv: 2010.15070[6] Grundmann, M., Neudecker, T., Hartenstein, H.: Exploiting Transaction Accumulationand Double Spends for Topology Inference in Bitcoin. In: Zohar, A., Eyal, I., Teague,V., Clark, J., Bracciali, A., Pintore, F., Sala, M. (eds.) Financial Cryptography andData Security. Lecture Notes in Computer Science, vol. 10958, pp. 113–126. SpringerBerlin Heidelberg (2019)[7] Heilman, E., Kendler, A., Zohar, A., Goldberg, S.: Eclipse Attacks on Bitcoin’s Peer-to-Peer Network. Tech. Rep. 263 (2015), https://eprint.iacr.org/2015/263 [8] Miller, A., Litton, J., Pachulski, A., Gupta, N., Levin, D., Spring, N., Bhattacharjee,B.: Discovering bitcoin’s public topology and influential nodes (2015)[9] Nakamoto, S.: Bitcoin: A Peer-to-Peer Electronic Cash System. Tech. rep. (2008)[10] Naumenko, G., Maxwell, G., Wuille, P., Fedorova, A., Beschastnikh, I.: Erlay: EfficientTransaction Relay for Bitcoin. In: Proceedings of the 2019 ACM SIGSAC Conference onComputer and Communications Security - CCS ’19. pp. 817–831. ACM Press, London,United Kingdom (2019). https://doi.org/10.1145/3319535.3354237, http://dl.acm.org/citation.cfm?doid=3319535.3354237 [11] Neudecker, T., Andelfinger, P., Hartenstein, H.: Timing Analysis for Inferring theTopology of the Bitcoin Peer-to-Peer Network. In: 2016 Intl IEEE Conferences onUbiquitous Intelligence Computing, Advanced and Trusted Computing, Scalable Com-puting and Communications, Cloud and Big Data Computing, Internet of Peo-ple, and Smart World Congress (UIC/ATC/ScalCom/CBDCom/IoP/SmartWorld).pp. 358–367 (Jul 2016). https://doi.org/10.1109/UIC-ATC-ScalCom-CBDCom-IoP-SmartWorld.2016.0070[12] Neudecker, T.: Security and Anonymity Aspects of the Network Layer of PermissionlessBlockchains. Ph.D. thesis (2019). https://doi.org/10.5445/IR/1000089033, https://publikationen.bibliothek.kit.edu/1000089033 [13] Park, S., Im, S., Seol, Y., Paek, J.: Nodes in the Bitcoin Network: Com-parative Measurement Study and Survey. IEEE Access , 57009–57022 (2019).https://doi.org/10.1109/ACCESS.2019.2914098, conference Name: IEEE Access[14] Tange, O.: GNU Parallel 20200522 (’Kraftwerk’) (May 2020), https://doi.org/10.5281/zenodo.3841377 [15] Wang, L., Pustogarov, I.: Towards Better Understanding of Bitcoin UnreachablePeers. arXiv:1709.06837 [cs] (Sep 2017), http://arxiv.org/abs/1709.06837http://arxiv.org/abs/1709.06837