OptChain: Optimal Transactions Placement for Scalable Blockchain Sharding
OOptChain: Optimal Transactions Placement forScalable Blockchain Sharding
Lan N. Nguyen, Truc D. T. Nguyen
CISE DepartmentUniversity of FloridaGainesville, FL, 32611Email: { lan.nguyen,truc.nguyen } @ufl.edu Thang N. Dinh
CS DepartmentVirginia Commonwealth UniversityRichmond, VA, 23284Email: [email protected]
My T. Thai
CISE DepartmentUniversity of FloridaGainesville, FL, 32611Email: [email protected]fl.edu
Abstract —A major challenge in blockchain sharding protocolsis that more than 95% transactions are cross-shard. Not onlythose cross-shard transactions degrade the system throughput butalso double the confirmation time, and exhaust an already scarcenetwork bandwidth. Are cross-shard transactions imminent forsharding schemes? In this paper, we propose a new shardingparadigm, called OptChain, in which cross-shard transactionsare minimized, resulting in almost twice faster confirmationtime and throughput. By treating transactions as a stream ofnodes in an online graph, OptChain utilizes a lightweight andon-the-fly transaction placement method to group both relatedand soon-related transactions into the same shards. At the sametime, OptChain maintains a temporal balance among shards toguarantee the high parallelism. Our comprehensive and large-scale simulation using Oversim P2P library confirms a significantboost in performance with up to 10 folds reduction in cross-shardtransactions, more than twice reduction in confirmation time, and50% increase in throughput. When combined with Omniledgersharding protocol, OptChain delivers a 6000 transactions persecond throughput with 10.5s confirmation time.
I. I
NTRODUCTION
Blockchain has emerged as a disruptive and transforma-tional technology, with great potential and benefits, offeringa promising new decentralized economy without the risk ofsingle point of failures, monopoly, or censorship [1]. Examplesof these systems are ranging from the cryptocurrency suchas Bitcoin [2] and Ethereum [3], to other infrastructures andapplication domains such as the Internet-of-Things [4], [5] andDigitial Health [6], [7]. Unfortunately, the performance levelof existing decentralized systems based on the Blockchaintechnology is too low to realize that vision, e.g., 7 transactionsper second (tps) and up to 60 minutes confirmation time forBitcoin and 10 tps/12 minutes for Ethereum.To this end, blockchain sharding , which splits theBlockchain into multiple disjoint parts, called shards, eachmaintained by a subgroup of nodes, has been proposed asa prominent solution for Blockchain scaling. Since each nodeonly needs to communicate with a few (hundreds) nodes fromthe same shard to maintain a small chunk of blockchain,sharding reduces substantially the storage, computation, andcommunication costs. This is different from legacy blockchainsystems, e.g., Bitcoin and Ethereum, in which all nodes needto communicate to maintain the same copy of blockchain,thus, the performance is limited by the nodes average pro-cessing capabilities. The latest sharding approaches such asOmniledger [8] and Rapidchain [9] can handle thousands of
The first two authors contribute equally to this paper. transactions per second with confirmation time about a fewdozens of seconds.All existing sharding approaches, however, face the samechallenge of handling cross-shard transactions, which involvedata from more than one shards. To prevent the double-spending problem [10], all shards that involve a cross-shardtransaction (tx) need to execute multiple-phase protocols toconfirm the tx. This can multiple fold increase both the latencyand the effort to confirm the tx, comparing to the case whenthe tx need to be processed by only one shard. In turns, extraefforts in confirming txs may lead to higher tx fees. To makeit even worse, more than 95% of the transactions are cross-shard [8], [9]. Previous sharding approaches often place txsinto shards randomly to balance the load among the shards.It is natural to ask “is there a smart transactions placementstrategy that reduces the cross-shard txs, making sharding evenfaster?”In this paper, we propose OptChain, a sharding-agnosticframework that boosts the performance of existing (and future)sharding approaches via optimizing the placement of txs intoshards. OptChain learns the pattern from the past txs to decidethe shard-location for incoming txs based on 1) whether suchplacement reduces the cross-shard txs and 2) load balanceamong the shards. Specifically, OptChain works on top anunexplored graph construction for transaction networks inUTXO model [2], termed
Transactions as Nodes (TaN), andintroduces a new score, termed
Temporal Fitness to assess thesuitability of placing an incoming transaction to each shard.The
TaN network is constructed by abstracting each trans-action as a node and there is a directed edge ( u, v ) if tx u usestx v as an input. This TaN is an online directed acyclic graph(DAG), in which nodes arriving one by one. This constructionis different from existing abstraction of transaction networksin which transactions are abstracted as edges among the nodesmade of addresses [11].The Temporal Fitness score is composed by two compo-nent scores
Transaction-to-Shard (T2S) and
Latency-to-Shard (L2S). The T2S scores between a transaction u and a shard S i , measures the probability that a random walk from anode/tx u ends up at some node in S i (see section IV.B),i.e., hence, how likely the transaction should be placed intothe shard without causing further crosshard txs. The L2S scoreestimates the processing delay when placing the transaction toa shard. Importantly, the protocol to estimate the two scoresis lightweight and is executed at the users side. Practicality.
Our solution
OptChain can be implemented a r X i v : . [ c s . CR ] J u l ith simple modification in user-side software, e.g., wallet.That is OptChain does not interfere with the core consensusprotocols and, hence, can be integrated into almost all shardingapproaches. Specifically, as computing the T2S score onlyrequires the information on the input txs, it can be doneefficiently at the user side by modifying the existing SimplePayment Verification protocol [2], i.e., users do not needto download the complete transaction history. Based on thelatencies observed from the shard, the wallet software at theuser side can use OptChain to make the decision on whichshard he/she wants the tx to be processed.To validate our approach, we measure the performance of anenhanced version of OmniLedger [8] with our OptChain ap-proach on existing Bitcoin transactions. The results indicatethat OptChain can effectively reduce the cross-shard txs upto 10 folds, cut the txs confirmation time by 93%, and at thesame time, increase the throughput by 50%. While we onlytest OptChain with Omniledger, we predict a similar level ofimprovement in performance when combining OptChain withother sharding protocols such as Rapidchain.
Our contribution.
We summarize our contribution as fol-lows. • We introduce a new way of sharding txs, reducing cross-shard txs via an optimal placement of txs into shards.This simple idea effectively reduces the cross-shard txsand boosts the performance comparing to the randomplacement in existing sharding approaches. • We investigate a new abstraction of transactions network(TaN) in which transactions are abstracted as nodes ratherthan edges among addresses in previous studies. This newabstraction results in an online DAG that can provide newways to analyze the transaction stream in UTXO-basedledgers. We provide various charateristics of this TaN onBitcoin transactions consisting of 298,325,121 nodes and696,860,716 edges. • We introduce a novel algorithm, called OptChain, thatanalyze the stream of txs in TaN network to make theoptimal placement of txs into shards. OptChain is simple,lightweight, and can be easily implemented into existingwallet software. • Our comprehensive experiments with an enhanced Om-niledger protocol on real Bitcoin transactions affirm thesignificant benefit of our approach in cutting down theconfirmation time and boosting throughput.
Organization.
The rest of this paper is structured as follows.We summarize the related work in Section II. Section IIIdiscusses the basis of handling cross-shard transactions inblockchain sharding and provides some observations as wellas primary goals of the transactions placement strategies.In Section IV, we investigate the TaN network and presentour OptChain algorithm. The experimental design and resultsare presented in Section V. Finally, Section VI gives theconcluding remarks. II. R
ELATED WORK
Blockchain sharding.
Several blockchain sharding proto-cols [8], [9], [12]–[15] have been proposed to address thescalability issue in legacy blockchain. In typical blockchainsharding, the entire state of the blockchain is splitted intopartitions called shards that contain their own independent piece of state and transaction history. The key idea is toparallelize the available computation power, dividing it intoseveral smaller shards where each of them processes a disjointset of transactions.Currently, most of existing sharding protocols are built ontop of the UTXOs model. The most notable exception isEthereum 2.0 [15] which is the next development phase ofthe Ethereum blockchain [3], employing the account model.Unlike the UTXO model, each transaction in the accountmodel has only one input and one output.The three core components of an existing sharding protocolare 1) how to (randomly) assign nodes into shards to formshard committees; 2) an intra-shard consensus protocol toexecute by shard committees (often a BFT protocol [8], [9]but also can be a Nakamoto-like protocol [16]); and 3) anatomic protocol for cross-shard transactions. In this work, wefocus on the third component in which we aim to optimize theplacement of transactions, hence mitigate the negative impactof cross-shard transactions on the performance of existingsharding approaches [8], [9].
Transaction networks.
In our method, we abstract therelation between transactions under a graph representation. Inprevious work, Kondor et al. [11] addressed the transactionsnetwork as a graph where each node represents a user address,each directed edge between two nodes is created if there isat least one transaction between the corresponding addresses.Our work is fundamentally different in which we abstracttransactions as nodes while an edge between two nodesrepresents the behavior that one transaction spends an outputof the other. By such representation, we model the problem oftransaction sharding to be an online graph partitioning problemwith temporal balancing, where nodes in a graph is dividedinto disjoint subsets and the objective is to identify whichsubset should contain a new arriving node.
Online and Balanced DAG Partitioning.
Online graphpartitioning has been addressed in the literature [17], [18].Stanton et al. [17] and Abbas et al. [18] have proposed multiplenatural, simple heuristics and compared their performance toMetis [19], a fast and offline method. The objective of theiralgorithms is to partition network vertices into almost equaldisjoint sets while minimizing number of crossing edges (edgewhose two endpoints are in two different set). However, theseworks are fundamentally different to our work, in which wewould like to minimize number of cross-shard transactionsrather than crossing edges. Furthermore, even their workseventually guarantee the balances between sets, we concernmore on temporal balancing, in which we would like thenumber of vertices on each sets should be almost equal forany given of time.III. C
ROSS - SHARD TRANSACTIONS AND T RANSACTIONS P LACEMENT S TRATEGIES
In this section, we present an overview of state-of-the-arttransaction sharding procedures. Although blockchain shard-ing itself could help boost the transactions confirmation pro-cess, nonetheless, the high amount of cross-shard transactionsis one of the obstacles that hinder the system from gettingbetter performance. Hence, we illustrate its negative impactwhich motivates us to devise the OptChain to tolerate thisissue.ig. 1: Handling cross-shard transactions in Omniledger [8]
A. Atomic Commit Protocols for Cross-shard Transactions
As both cross-shard protocols in [8] and [9] are built on topof ledgers using UTXO model, we begin with the presentationof this model.
Unspent Transaction outputs (UTXO) model.
The UTXOmodel was introduced in original Nakamoto protocol forBitcoin [2]. In this model, transactions may have multipleinputs and outputs where an output is assigned with creditsand locked to a user’s address. This newly created transactionoutput is referred to as a UTXO. The UTXOs may then beused as inputs of another transaction, and after this transactionis committed to a block, those UTXOs will be marked as spent and cannot be used again. For simplicity, we will consider atransaction tx with two inputs tx in , tx in . Cross-shard Transactions.
Let S in , S in , and S out denotethe shards that contain tx in , tx in and tx , respectively. If all theshards S in , S in , and S out are the same, we have a same-shardtransaction, otherwise the transaction is cross-shard (cross-TX).If transactions are placed into shards randomly, it was shownthat the probability for a typical transaction having two inputsand one output to be a cross-shard transaction is about 94%[8], assuming 4 shards, and 99.98%, assuming 16 shards. Committing Cross-TXs in OmniLedger [8]. OmniLedgerproposed a novel atomic protocol to commit cross-TXs con-sistently among all shards. The protocol locks all inputtransactions at the input shards before committing the outputtransaction(s) to output shard(s).1)
Initialize . A user creates a cross-TX tx whose inputsspend UTXOs, e.g., tx in , tx in , from some input shards,e.g., S in , S in . The client gossips the cross-TX and iteventually reaches all input shards.2) Lock . All input shards validate the transactions withinhis shard. If the transactions are valid, they are markedspent on the shard’s ledger, and a proof-of-acceptance is gossiped; otherwise a proof-of-rejection is gossiped.3)
Commit . If all input shards gossip the proof-of-acceptance, the client can gossip an unlock-to-committransactions that eventually reach all output shards. Inturn, each output shard validates the transaction andincludes it to his ledger. However, if even one inputshard issued a proof-of-rejection, then the transactioncannot be committed and has to abort. The client thencan gossip an unlock-to-abort message to reclaim thefund.
Committing Cross-TXs in RapidChain [9]. Rapidchainuses a “yanking” mechanism, in which the input transac-tions, e.g., tx in , tx in , will be first moved to from the inputshards, e.g., S in , S in , to the output shard, e.g., S out via an inter-committee protocol. After all the input transactionsare successfully “yanked” to the output shard, then the finaltransactions can be added to the ledger of the output shard. B. Performance Penalty for cross-TXs.
Comparing to same-shard transactions, cross-TX incursmuch higher confirmation time, communication and compu-tation costs.
Longer confirmation time.
In Omniledger [8], a cross-TXwill easily double the confirmation time of those in the same-shard transactions. For the same-shard transaction, the useronly needs to submit the transaction to the shard and wait forconfirmation. The confirmation time will only be the roundtime trip between the users and the shard pluses the time forthe shard committee to agree on the transactions. A cross-TX will incur extra time to confirm the input transactions asthe input shards as well as another round time trip betweenthe users and the shard committees. The same is applied forRapidchain cross-TX protocol as each cross-TX incurs extraround-time trip among the committees as well as the waitingtime for input transactions to agree on ‘yanking’ transactionsbetween shards.
Extra communication and computation cost.
For a typicalcross-TX with 2 inputs and one output, the communicationcost will triple that of a same-shard transaction as all the threeshard committees and the user need to communicate to confirmthe transaction. The same holds for the computation cost.Thus, assuming uniform time and cost to handle transac-tions in shards, each cross-TX will double confirmation time and triple the bandwith consumption and computation cost .Therefore, if we can reduce the fraction of cross-TXs to 20%,we will cut the confirmation time by more than 40% and morethan double the throughput.
C. OptChain: A Transactions Placement Strategy
Random Placement . In existing sharding approaches [8],[9], transactions are placed randomly into shards. Often, thehashed value of a transaction is used to determine whichshards the transaction will be placed into. This will balancethe amount of transactions per shard, however, cause almost alltransactions to be cross-TXs. For a typical transaction havingtwo inputs and one output to be a cross-shard transaction isabout 94%, assuming 4 shards, and 99.98%, assuming 16shards [8]. To be specific, the systems do not consider therelationship between transactions on sharding process, whichmakes a majority of transactions eventually become cross-TXs.
Smart Transaction Placement.
Ideally, the best method isto groups well-connected transactions into a same shard. Bythat way, we can minimize the number of verification steps(step 2) of cross-TXto get proof-of-acceptance. Moreover, thecurrent state of shards should also be considered. We wouldlike to avoid situations where some shards are extremelybusy (i.e huge number of transactions in queue to wait forverification) while some are idle. Intuitively, with a randomselection, we can balance the number of transactions in eachshards. However, some transactions could take more time toprocessed than the others. Therefore, in simulation, we observethat there are some certain moments the random selection willeventually cause extremely imbalance on queue sizes betweenshards. ptChain.
Motivated by observations on the limits ofOmniLedger and RapidChain, in this paper, we investigatean optimization problem in which users need to determinethe best shard to submit their transactions in order to min-imize cross-TXs while guaranteeing the temporal balance,thus, shorten the confirmation time and boost the overallsystem throughput. Specifically, the ultimate goal of our smarttransactions placement OptChain are:1)
Fast Confirmation Time : As major txs are same-shardtxs and load are distributed evenly, txs get confirmed inmuch shorter time.2)
High Throughput : Same-shard txs require less time andcommunication to confirm, thus, the system throughputgets significantly boosted.The above goals are obtained via optimizing two indirect goals1)
Cross-TX Minimization : Reduce the number of cross-shard transactions by grouping related transactions intoa same shard.2)
Temporal Balancing : To distribute load evenly amongshards to increase parallelism and reduce queuing time.In practice, we aim to deploy OptChain as a user-sidesoftware. By monitoring its own transactions as well as theinformation on the loads and confirmation time at the shards,a client software can make decision on which is the best shardto submit transactions. It is important that users do not haveto store the whole blockchain to optimize the placement.In the next section, we will propose a lightweight, yet,efficient solution, answering the question:
How to identify anappropriate shard for a new transaction such that in the longfuture, our system can achieve four main goals as described?
IV. O PT C HAIN A LGORITHM
In this section, we propose an algorithm used by OptChainto place transactions into shards. First, we introduce the systemmodel, in which we represent transactions under a directednetwork. Under this model, we utilize a well-known PageRankanalysis to propose T2S-score, which is to measure how likelya transaction should be placed into the shard. Then, we proposea mathematical model to estimate confirmation latency forplacing a transaction into shards, which we called L2S score.Finally, we describe how OptChain places transactions intoshards based on the combination of T2S and L2S scores.
A. Transaction-as-Node Network and Partitioning
To observe the relation between transactions, we modelthe set of transactions under a graph representation, whichis defined as follows:
Definition 1 (TaN Network):
A TaN network of a set oftransactions is presented as a directed graph G = ( V, E ) where V is the set of transactions and E is a set of directed edgesin which there exists ( u, v ) ∈ E if the transaction u uses theUTXO(s) of transaction v .To have a clear picture on how a TaN network looks like, weconstruct a TaN network from a set of transctions in the first508,241 blocks of the Bitcoin blockchain which was gatheredby MIT [20]. The most recent transaction of the dataset wasissued on February 2018. From this dataset, we construct aTaN network of 298,325,121 nodes and 696,860,716 edges.Nodes that do not have any outgoing edges represent coinbasetransactions (rewards for mining Bitcoin blocks). On the other hand, nodes that do not have any incoming edges representtransactions whose UTXOs have not been spent. TaN networkis a directed acyclic graph since a transaction only usesUTXO(s) of past transactions. Therefore, TaN network can besorted in a topological order, which exactly reflects the orderof appearance of transactions.Fig. 2a shows the plot of the degree distribution of the con-structed TaN network in log-log scale. The degree distributionof TaN network exhibits power-law distribution with averagein- or out- degree of ≈ . . There are total 508,241 coin-base transactions (transactions without inputs), and 369,2947transactions whose UTXO(s) have not been spent. Also, thereare 37,108 transactions without any outputs or inputs. As canbe seen from the degree distribution in Fig. 2b, major nodes(93.1%) in TaN network have the in-degree lower than 3. Asregards the out-degree, 97.6% of nodes have the out-degreelower than 10, and 86.3% lower than 3.We investigate the average degree of nodes of TaN net-work overtime and plot in Fig. 2c. We observe that for amajor of time, the average degree of TaN network is stableand consistent. There are only 2 periods the average degreesignificantly changes, which is the first 1 millions transactionand around 80,000,000th one. This behavior does not happenregularly because: (1) the first period is that the system needsto generate a lot of coinbase transactions for funding and (2)there was a flooding attack at the second period [21], whichcauses mining pools to create a lot of transactions with highdegree to clean up “trash” transactions.By distributing transactions into shards, the task of transac-tions sharding eventually become partitioning the TaN networkinto k disjoint subsets of nodes S = { S , ..., S k } , where k isthe number of shards and S i denotes a set of transactions undermanagement of shard i . We have ∪ ki =1 S i = V and S i ∩ S j = ∅ for all i, j = 1 → k . For simplicity, we also call S i as theshard i .Given a transaction represented by node u ∈ V , denote S ( u ) ∈ S as a shard containing u . Let S in ( u ) as a set ofinput shards of u . Then u is a cross-TX iff S in ( u ) (cid:54) = { S ( u ) } .Therefore, we consider the task of distribute a new transactioninto shards as an online partitioning to the TaN network. To bespecific, given a TaN network G = ( V, E ) , a set of k disjointsubsets of nodes S = { S , S , ..., S k } and a new arrivingtransaction (node) u , our task is to identify S ( u ) ∈ S . B. Transaction-to-Shard Score
In this part, we will propose a metric to measure the fitnessscore between the new arrival transaction and shards, whichwe called Transaction-to-Shard (T2S) score. This score ismotivated by a well-known graph measurement, called PageR-ank. To be specific, PageRank assigns numerical weighting toeach node of a network with the purpose of measuring noderelative importance within the network. A PageRank vectoris a weighted sum of the probability distribution obtainedby taking a sequence of random walk steps starting from aspecified initial distribution. PageRank has been proven to bean efficient method on graph local partitioning, in which nodesin a same set of partition tend to have a similar weight.Although regular PageRank computation could cost consid-erable runtime, we utilize the trait that TaN is acyclic andthe order of nodes’ appearance is also TaN’s topological orderto devise a fast T2S-score computation of each nodes. To be a) Degree distribution (b) Cumulative distribution (c) Changes in average degree
Fig. 2: TaN network statisticsspecific, we represent the T2S-scores of a transaction u toshards under a k -dimensional vector p ( u ) , in which the entry i measures the fitness between u and shard i . Denote r [ i ] asthe value of entry i of vector r . Also, let N in ( u ) ( N out ( u ) )as a set of input (output) transactions of u . p ( u ) is computedby the following equation: p ( u ) = αs ( u ) + (1 − α ) (cid:88) v ∈ N in ( u ) p ( v ) | N out ( v ) | where α is a constant in (0 , and s ( u ) is a starting vectorfor a node u . s ( u ) is initiated as follows: • If u is a new arriving node, s ( u ) = { } k • Otherwise, s ( u ) receives a value | S ( u ) | at the entrycorresponding to shard S ( u ) and elsewhere.Clearly p ( u ) can easily be calculated in the manner similarto Bread-First Search, in which we start from nodes who haveno input (coinbase transaction). Therefore, computing p ( u ) for all u ∈ V costs O ( k ( | V | + | E | )) runtime complexity,which is very expensive as the TaN network grows. However,we observe that: after placing a new transaction u into shard S ( u ) , the change on s ( v ) for all v ∈ V does not impact thefitness score of all nodes other than normalization scale. Tobe specific, assume at time t + 1 , u arrives and is placed intoshard i ( S ( u ) = S i ). Denote p ( t ) ( v ) is the T2S-score vectorof node v at time t and S ( t ) i is the shard i at time t . We have: • If v (cid:54) = u , p ( t +1) ( v )[ i ] = | S ( t ) i || S ( t ) i | +1 p ( t ) ( v )[ i ] and p ( t +1) ( v )[ j ] = p ( t ) ( v )[ j ] for all j (cid:54) = i . • If v = u , p ( t +1) ( v )[ i ] = α | S ( t ) i | +1 + S ( t ) i S ( t ) i +1 p ( t ) ( v )[ i ] and p ( t +1) ( v )[ j ] = p ( t ) ( v )[ j ] for all j (cid:54) = i .Therefore, rather than computing p ( v ) for all v ∈ V fromscratch to get T2S score of a new transaction u , we proposethe following methods for the faster calculation. First, weintroduce two other vectors p (cid:48) ( v ) , s (cid:48) ( v ) associated to eachnode v ∈ V in addition to s ( v ) and p ( v ) . If a transaction v is placed in shard j then s (cid:48) ( v )[ j ] = 1 and s (cid:48) ( v )[ i ] = 0 forall i (cid:54) = j , s (cid:48) ( v ) is fixed right after v is placed. Given a newtransaction u , p ( u ) can be computed as follows. • If u is a coinbase transaction, there is no calculationneeded. • Otherwise, we set p (cid:48) ( u ) = (1 − α ) (cid:80) v ∈ N in ( u ) p (cid:48) ( v ) | N out ( v ) | .The T2S score of node u is p ( u ) = { p (cid:48) ( u )[ i ] (cid:80) v s (cid:48) ( v )[ i ] } ki .Then, after placing u into shard i ( S ( u ) = S i ), we update p (cid:48) ( u ) = p (cid:48) ( u ) + αs (cid:48) ( u ) . Overall, the computation of p ( u ) nowonly costs O ( | N in ( u ) | k ) . As TaN network has been shown tobe scale-free, the average computation, thus, costs only O ( k ) . Discussion.
Why do we develop the new T2S score insteadof using existing graph partitioning methods to minimizethe number of cross-TX? The best way yet unrealistic tominimize the number of cross-TXs is that we know thestructure of the TaN network beforehand, i.e. all transactionshave been arrived, and apply graph partitioning algorithms.But even those, does such algorithm improve the performanceof sharding blockchain? The answer could be not likely.In experiment, we applied a well-known graph partitioningtool, called
Metis k -way [19], to get the partitioning ontransactions. Metis aims at partitioning the graph into disjointsets of nodes with almost equal size while minimizing thenumber of edges whose two endpoints belong to different sets.However, when simulating, if we put transaction exactly like inthe Metis solution, the system’s throughput and confirmationlatency are highly impacted as the Metis solution tends to putlarge amount of consecutive transactions into one shard. Moredetails of such experiments will be shown in Section V.A more realistic method, but simple, is Greedy . To comparewith Metis, given n txs and k shards, we set the maximumnumber of transactions that a shard can have as (1 + (cid:15) ) (cid:98) n/k (cid:99) .Then we consider transactions sequentially. Considering atransaction u arrives at a moment t , denote S ( t ) i as a setof transactions in shard i at time t . The greedy algorithmcomputes the cost of placing u on shard j as f ( u, j ) = |S in ( u ) \ S ( t ) j | . Then the algorithm places u into a shard j which has the maximum f ( u, j ) and its size has not exceeded (1 + (cid:15) ) (cid:98) n/k (cid:99) . Intuitively, the greedy solution will help reducethe number of cross-TXs. However, this solution does nottake the global view on TaN network structure while onlyconsidering connection of one-hop away from u . Thus, inthe long future, the performance of greedy algorithm becomesundesireable.To prove the efficiency of T2S-score, we compare thepercentage of cross-TXs between Omniledger, Greedy solutionand a solution which is similar to Greedy except that aftercomputing p ( u ) , we put u into a shard with the highest T2S-core, i.e. argmax i p ( u )[ i ] . We set α = 0 . and we call suchmethod T2S-based . With both Greedy and T2S solution, weset (cid:15) = 0 . .We use the same data set from MIT. First, we ran thealgorithms from scratch where all shards are empty at thebeginning. The result is presented in Table I. It is easy to seethat our solution significantly reduces the number of cross-TXs and overcomes Greedy solution with a huge margin. EvenMetis is unrealistic, we put its results in the table as a baselinefor comparison.TABLE I: Percentage of cross-TXs when running from scratch k Metis Greedy Omniledger T2S-based4 1.66 % 24.62 % 80.82 % 9.28 %8 3.09 % 27.02 % 90.33 % 12.52 %16 4.70 % 28.14 % 94.87 % 15.73 %32 6.91 % 28.69 % 97.09 % 18.94 %64 9.91 % 28.97 % 98.18 % 21.65 %
Next, we consider at a certain moment, the system alreadyplaces a certain amount of transactions into shards. We thenapply the algorithms with a set of new arrival transactionsand compare the number of cross-TXs in such set. To bespecific, first, we use Metis to partition the TaN network of 30millions Bitcoin transactions into k shards. Then we apply thealgorithm to place transactions of a sequence of next 1 millionstransactions into k shards. The results is presented in TableII. Again, our solution showed its efficiency comparing withGreedy and OmniLedger in term of minimizing the numberof cross-TXs. C. Latency-to-Shard Score and Temporal Fitness
We have shown that a simple solution using T2S-scorecould significantly reduce the number of cross-TXs. However,this is not a ultimate goal of our algorithm design. What ifthere is a huge amount of transactions with the same “fittest”shard w.r.t T2S-score arriving sequentially? Trivially, puttingall those transactions into the same shard is not a goodsolution. Therefore, in this part, we propose a mathematicalmodel to estimate confirmation latency of a transaction undertransaction sharding. We call the estimated latency
Latency-to-Shard (L2S) score. Hence, the best transaction placementis the one that maximizes the T2S-score while minimizing theL2S-score.Let consider a moment a new transaction u arrives andsystem shards are S , ...S k . Assume if u is placed in shard j , u will need proof-of-acceptance from a set of shards S j . We model the communication time between a user whocreates u and the shard S i under exponential distribution l ( i ) c ( t ) = λ ( i ) c e − λ ( i ) v t , where λ ( i ) c is expected communicationtime, which could be collected through frequently samplingbetween the user and shard S i . Also, for each shard S i , wemodel the verification time of S i under exponential distribution l ( i ) v ( t ) = λ ( i ) v e − λ ( i ) v t , where λ ( i ) v is expected verification time,which could be estimated from observation of recent consensustime of shard i and its current queue size. With high precision,it is likely that λ (1) v (cid:54) = ... (cid:54) = λ ( k ) v (cid:54) = λ (1) c (cid:54) = ... (cid:54) = λ ( k ) c Therefore, a probability distributed function of time to getproof-of-acceptance from shard S i is modeled as follows. TABLE II: Number of cross-TXs when running from a certainstage of the system k Greedy Omniledger T2S-based4 335,269 837,356 112,6578 407,747 922,073 172,97816 441,267 960,935 226,17132 449,032 979,323 282,10864 454,321 988,144 366,854
Algorithm 1
Transaction Sharding in OptChain
Input : G ( V, E ) , S , ...S k , p (cid:48) : V → R + , α and a newtransaction u Output : a new state of S , ...S k u p (cid:48) ( u ) = (1 − α ) (cid:80) v ∈ N in ( u ) p (cid:48) ( v ) | N out ( v ) | p ( u ) = { p (cid:48) ( u )[ i ] | S i | } ki =1 u for j = 1 → k do E ( j ) = (cid:82) ∞ t (cid:82) t f ( j ) v ( x ) f ( j ) v ( t − x )∆ x ∆ t end for u into shard with the highest Temporal Fitness score s u = argmax i (cid:16) p ( u )[ i ] − . · E ( i ) (cid:17) S s u ← S s u ∪ { u } Update after placing u into S s u p (cid:48) ( u )[ s u ] = p (cid:48) ( u )[ s u ] + α Return S , ...S k f ( i ) ( t ) = (cid:90) t l ( i ) c ( x ) l ( i ) v ( t − x )∆ x = λ ( i ) c λ ( i ) v λ ( i ) v − λ ( i ) c (cid:16) e − λ ( i ) c t − e − λ ( i ) v t (cid:17) while the cumulative distributed function is: F ( i ) ( t < T ) = (cid:90) T λ ( i ) c λ ( i ) v λ ( i ) v − λ ( i ) c (cid:16) e − λ ( i ) c t − e − λ ( i ) v t (cid:17) ∆ t = λ ( i ) v λ ( i ) v − λ ( i ) c (1 − e − λ ( i ) c T ) − λ ( i ) c λ ( i ) v − λ ( i ) c (1 − e − λ ( i ) v T ) As the user can send request for verification simutaneouslyto shards in S j , the probability that verification process is doneby time T is computed as F ( t < T ) = (cid:89) S i ∈S j F ( i ) ( t < T ) Thus, the probability distributed function of time for theuser to get all proof-of-acceptance if place u into shard j ismodeled as follows f ( j ) v ( t ) = ∆ F ( t )∆ t = (cid:88) S i ∈S j λ ( i ) c λ ( i ) v λ ( i ) v − λ ( i ) c (cid:16) e − λ ( i ) c t − e − λ ( i ) v t (cid:17) (cid:89) S r ∈S j \ S i F ( r ) ( t ) imilarly, we can find the probability distribution for u toget confirmation from shard j , which is: f ( j ) c ( t ) = λ ( j ) c λ ( j ) v λ ( j ) v − λ ( j ) c (cid:16) e − λ ( j ) c t − e − λ ( j ) v t (cid:17) Therefore, the L2S-score of u if putting into shard j iscomputed by E ( j ) = (cid:90) ∞ t (cid:90) t f ( j ) v ( x ) f ( j ) v ( t − x )∆ x ∆ t In overall, for a given transaction, we would like to balancebetween the T2S and L2S-score. Specifically, given a newarrived transaction u and a shard j , we define the TemporalFitness score between u and j as p ( u )[ j ] − . · E ( j ) .OptChain then places u into the shard whose has the highestTemporal Fitness score. The procedure of transaction shardingin OptChain is presented in detailed by Alg. 1.V. E XPERIMENTS
The aim of this experiment is to evaluate the impact ofOptChain on Blockhain system’s latency and throughput. Todemonstrate the strengths of our proposed solution, the per-formance of OptChain is rigorously compared to OmniLedger,Metis, and the Greedy heuristic discussed in section IV.
A. Experiments Design and Configuration
The experiments presented in OmniLedger [8] used the first,arguably simple, 10,000 Bitcoin blocks which contain only10,093 transactions in total. Additionally, of those transactions,99.1% are coinbase, which cannot be cross-TXs. At that pointin time, Bitcoin was still in an initial phase in which therewas only about 1 transaction per block and most of them werecoinbase transactions. Therefore, their experimental results donot represent the impact of cross-TXs.Our experiments are conducted on the first 10 millionBitcoin transactions, taken from the dataset in [20]. Fromthis dataset, we build a TaN network that contains 10,000,000nodes and 19,958,051 edges. As opposed to OmniLedger, ourdataset is much bigger with more chance of creating cross-TXs. Specifically, with 16 shards, the OmniLedger’s randomplacement produces 9,349,979 cross-TXs comprising 93.5%of the whole set.We develop a sharding-based blockchain simulation and runthe transactions placement algorithms to calculate the latencyand throughput. Particularly, we use the OverSim framework[22] to simulate a Bitcoin-like blockchain system on OM-NeT++ 4.6 [23], which is a discrete event-based networksimulator. The bandwidth of all connections between nodesis set to 20 Mbps and a latency of 100 ms is imposed on allcommunication links. We set the block size of 1MB becausethis is currently the block size limit in Bitcoin. The averagesize of a transaction is about 500 bytes, so we put about 2000transactions in one block. A shard is assigned with about 400validators and one leader that are randomly placed at differentcoordinates. In our simulation, the distance between nodesaffects the communication latency. Each shard implements aqueue (or mempool) to store incoming transactions that havenot been processed yet.Our simulation involves a set of clients that continuouslyissue transactions from the dataset to the system at a predefined TABLE III: Experiment configuration
Number of transactions 10,000,000Block size 1 MBTransactions per block 2,000Network bandwidth 20 MbpsNumber of shards 4, 6, 8, 10, 12, 14, 16Transactions rate (tps) 2000, 3000, 4000, 5000, 6000Algorithms OptChain, Metis k-way, OmniLedger, Greedy rate. We re-implement the mechanism for handling cross-TXsas decribed in Section III. Before sending a transaction u for verification, a client will run a transactions placementalgorithm to determine the shard S ( u ) to place that transaction. Getting rid of Omniledger’s bottleneck.
In Omniledger,users gossip each transaction to all nodes in the network, thus,all nodes must receive the whole blockchain (and also theunconfirmed transactions, if any). Thus, even when the numberof nodes and shards go to infinity, the throughput of nodesin Omniledger is still limited by typical network bandwidthof the nodes. To address this issue, users will direct eachtransaction directly to the input shards and output shards ofthe transaction.
Transaction placement strategies.
We compare OptChainwith three transaction placement methods. • OmniLedger: The default random placement strategy inOmniledger. • Greedy: Greedy placement strategy, discussed in Sec-tion IV-B. • Metis k -way: Running (offline) Metis partitioning algo-rithm to partition the network into k shards.As Metis k -way is an offline algorithm, it is not a realistictransactions sharding scheme. However, if we can put trans-actions as in Metis solution, we can minimize the numberof cross-TXs. Our purpose is, therefore, to see whether onlyminimizing the number of cross-TXs can improve the systemperformance. Therefore, we first input the whole TaN networkto get its Metis solution and then use the resulting partitionsto determine S ( u ) for each transaction u .We carry out the experiment with various configurations bycombining different number of shards and transaction rates.Specifically, we vary the number of shards from 4 to 16to conform with the experiments presented in OmniLedger[8]. With regard to the transactions rate, which represents therate at which transactions are sent to the system, we refer tothe VISA-level throughput of 4000 transactions per second(tps) to set the transactions rate from 2000 tps to 6000 tps.Table III summarizes all the parameters and configuration ofthe experiments. B. Experimental results
In Fig. 3, we summarize the system’s average latencyand throughput when running the algorithms with differentcombination of transactions rates and number of shards. Thethoughput is calculated by taking the number of transactiondivided by the total time for all transactions get committed.The latency of a transaction is measured by the time fromwhen the transaction is sent until it is committed to theblockchain. In this part, we would like to have more insighton how different configuration impacts the performance ofOptChain as well as other transaction sharding methods interm of throughput and latency. a) OptChain(b) OmniLedger(c) METIS k-way(d) Greedy
Fig. 3: Impact of different transactions rates and number ofshards on the latency and throughput
1) Maximum Throughput:
From Fig. 3, it is easy to see thatall the methods achieve their highest throughput at transactionrate of 6000 and 16 shards. However, except OptChain, theother three algorithms produce throughput much lower thanthe input rate, which shows that these three are incapableof handling such transaction rate in this setting. We observethat even OptChain does not always run well in all config-urations, i.e. throughput is lower than transaction rate. Butfor each value of transaction rates, OptChain always has acertain configuration on the number of shards guaranteeing nobacklogging in the system. To be specific, with a transactionrate of 2000, OptChain is totally healthy with at least 6 shards.This number in transaction rates of 3000, 4000, 5000, 6000 is8, 10, 14, 16 respectively. Meanwhile, the Omniledger systemneeds at least 16 shards to be able to process up to 3000 (a) Number of shards = 16 (b) Varying transactions rate and
Fig. 4: System throughputtransactions per second. With 16 shards, Greedy is only ableto process with transactions rate up to 5000. Therefore, givena same transaction rate, OptChain needs a lower number ofshards than other methods to avoid backlogging in the system.For a more comprehensive comparison, Fig. 4b presentsthe maximum system throughput at different pairs of valueof transactions rate and number of shards. As can be seen, noother transaction sharding methods can reach up to the samelevel as OptChain. For example, the maximum throughputOptChain can achieve at 16 shards is 34.4%, 30.5%, and16.6% higher than that of OmniLedger, Metis, and Greedy,respectively.Metis has been proven to be the best method, yet unrealistic,to distribute transactions into shards in order to minimize thenumber of cross-TXs. High number of cross-TXs is claimed byOmniledger [8] to be the main factor limiting the system per-formance. However, when we place transactions as in Metis’ssolution, the throughput never inline with transaction rate. Forexample, Fig. 4a shows the throughput of the algorithms aswe fix the number of shards to 16 and vary the transactionsrates. As previously stated, OptChain, Omniledger and Greedyare comfortable to a certain rate within such range. ButMetis’s throughput never reaches the transaction rate. Thus,we conclude that high number of cross-TXs is not a sole factorhindering Blockchain sharding performance.
What cause Metis’s throughput being worse than otherthree solutions? We take more insights on the timeline thattransactions are finally committed. We set the transaction rateto be 6000 and 16 shards. Then we count the number oftransactions getting committed in each 50-seconds period andplot the results as in Fig. 5. It is easy to see that OptChain,OmniLedger, and Greedy produce almost consistent numberof committed transactions over each period of 50 seconds.Meanwhile Metis is not efficient during the first 500 seconds.Also, Metis tends to agitate more than other three solutions,which could imply congestion on shards in some certainmoments, i.e. some shards get more transactions than the other.The huge drop in the end of each line in Fig. 5 is because thesimulator has reached the end of the dataset in which no moretransactions are sent to the system.To observe the congestion, we plot out the change on queuesizes of shards under four sharding methods as in Fig. 6.The congestion on Metis method is from the fact that Metistends to put a large amount of consecutive transactions intoone same shard. Thus, the pattern, in which some shards areoverwhelmed while the others have no transactions, happensfrequently in Metis. Greedy also met a situation that someig. 5: Number of committed transactions across timeshards have no transaction in several moments but in over-all, transactions are splitted more equally than Metis amongremaining shards. Such event does not happen in OptChainand OmniLedger. However, as OmniLedger is not capable ofrunning with a transactions rate of 6000 tps, shards queuesize will increase linearly overtime. OptChain performs thebest in term of load balancing among shards. We can seethat both the maximum and minimum queue size in OptChainare consistent and stable. Also, in worst case, a queue sizein OptChain only reaches up to ≈
2) Transaction Latency:
Next, we compare the average(maximum) transaction latency, i.e., transaction confirmationtime , among the different sharding approaches. As demon-strated in Fig. 3, all four sharding methods share the samebehavior: at a certain transaction rate, the average transactionlatency decreases significantly when the number of shardsincreases. Therefore, all four methods performs their best interm of latency with the configuration of 16 shards as thenumber of transactions in each shard is low enough.To closely evaluate the latency, we varied the transactionrate while keeping the number of shards at 16 as shownin Fig. 8a. Clearly there exists a considerable gap be-tween OptChain’s average latency and other methods’ results.OptChain always achieves the best performance comparingto the others. In fact, at the transactions rate of 4000 tps,the system only takes 8.7 seconds to process a transactionin average. At low transactions rate (i.e., 2000 to 3000 tps),we can see that all algorithms except for Metis have goodlatency in general. This is because the system can balancebetween throughput and transaction rate at these ranges, thusno backlogging happens.As we increase the rate to 6000 tps, our algorithm stillmaintains a good latency, while we can see a significantincrease in other algorithms. In particular, at 6000 tps, wereduce up to 93% the latency comparing to the OmniLedger. (a) OptChain (b) Omniledger(c) Greedy (d) Metis k-way
Fig. 6: Maximum and minimum queue size of shards overtime Fig. 7: Queue size ratioThis comes from the fact that OmniLedger can not tolerate6000 transactions per second in such settings. Thus, thequeuing delay (time staying in queue) of a transaction willincrease overtime and come to infinity. This behavior can alsobe observed in Greedy. As for Metis, even though it has theleast amount of cross-shard transactions, it still gets really highaverage latency. This issue can be explained as Metis failedto achieve the temporal balance in queue size between shardsas we mentioned above, causing there were only some activeshards at a time and exacerbating the final average latency.Next, we measure the system’s latency with different com-binations of transactions rate and number of shards as inFig. 8b. Specifically, we set the configuration in the same wayas Fig. 4b. Again as can be seen in Fig. 8b, OptChain obtainsthe best performance in term of balancing throughput andtransaction rate. At these configurations, the highest averagelatency of OptChain is only 10.5 seconds at the transactionsrate of 6000 tps and 16 shards, while OmniLedger reaches346.2 seconds at this configuration. In addition, at the trans-actions rate of 4000 and 10 shards, OptChain reduces up to98% the latency as compared to OmniLedger. a) Number of shards = 16 (b) Vary transactions rate and
Fig. 8: Average transaction latency (a) Number of shards = 16 (b) Vary transactions rate and
Fig. 9: Maximum transaction latencyFig. 9a illustrates the maximum latency to process transac-tions with 16 shards at different transaction rates. At 6000 tps,OptChain takes at most 100.9 seconds for a transaction whileOmniLedger, Metis, and Greedy respectively takes 1309.5,1345.9, and 628.9 seconds. This result, together with Fig. 8,again confirms that our algorithm significantly reduces thetransaction processing latency. Additionally, in Fig. 9b, wealso plot the result of this metric at different combinationsof transactions rate and number of shards as in the previousFig. 8b. Among these configurations, the highest latency weever reach is 102.7 seconds at the transactions rate of 5000 tpsand 14 shards. At this point, OmniLedger, Metis, and Greedyrespectively takes at most 1397.0, 1730.0, and 497.0 secondsto process one transaction.For a more detailed view on the impact of algorithmson system’s latency, in Fig. 10, we present the cumulativedistribution of the latency when we set the shards numberto 16 and transactions rate 6000 tps. As can be seen, up to70% of the transactions are processed within 10 seconds withOptChain. For other algorithms, within this time frame, only41.2%, 7.9%, and 2.4% of the transactions were completedwith Greedy, OmniLedger, and Metis, respectively.
C. Summary of results
The experiments conducted in this section notably showthat the proposed OptChain significantly outperforms othertransaction sharding methods. To be specific, OptChain couldreduce up to 93% the latency and increase 50% the throughputin comparison to OmniLedger. Furthermore, OptChain is morescalable than OmniLedger where it can scale up to the rate of6000 transactions per second and 16 shards. Meanwhile, fora certain transaction rate, OptChain requires less shards thanOmniLedgers to guarantee the system performance withoutbacklogging. In overall, Optchain’s good performance comes Fig. 10: Latency distributionFig. 11: OptChain scalabilityfrom the fact that OptChain is able to concurrently handle twomain factors that hinder the system performance: (1) reducingcross-TXs and (2) temporally balancing workload betweenshards.The highest transaction rate OptChain can scale up to(throughput is equal to transaction rate) with multiple numberof shards is plotted out as in Fig. 11. The best throughput ofOptChain is almost linear with the number of shards and itcan reach above 20,000 tps with 62 shards. More importantly,when the throughput is comfortable with transaction rate,OptChain guarantees that the confirmation delay is never morethan 11 seconds. VI. C
ONCLUSION
Handling cross-shard transactions has been a major chal-lenge in research on blockchain sharding as they are themain factor that limits the scalability of the system. In thispaper, we have proposed OptChain, a scalable protocol tooptimally place transactions into shards, which guarantee re-ducing number of cross-shard transaction as well as temporallybalancing workload between shards. In experiments, OptChainreduces the latency by 93% and increases the throughput by50% in comparison with the state-of-the-art sharding system,OmniLedger. Finally, our empirical evaluation demonstratesthat OptChain scales smoothly with transactions rate up to6,000 transactions per seconds with 16 shards and potentiallyhandle much higher rate with more shards.A
CKNOWLEDGEMENTS
This work was supported in part by NSF EFRI-1441231,NSF CNS-1814614, and DTRA HDTRA1-14-1-0055.
EFERENCES[1] M. Swan,
Blockchain: Blueprint for a new economy . ” O’Reilly Media,Inc.”, 2015.[2] S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” 2008.[3] Ethereum, “ethereum/sharding.” [Online]. Available: https://github.com/ethereum/sharding/blob/develop/docs/doc.md[4] K. Christidis and M. Devetsikiotis, “Blockchains and smart contracts forthe internet of things,”
Ieee Access , vol. 4, pp. 2292–2303, 2016.[5] S. Huckle, R. Bhattacharya, M. White, and N. Beloff, “Internet of things,blockchain and shared economy applications,”
Procedia computer sci-ence , vol. 98, pp. 461–466, 2016.[6] X. Yue, H. Wang, D. Jin, M. Li, and W. Jiang, “Healthcare datagateways: found healthcare intelligence on blockchain with novel privacyrisk control,”
Journal of medical systems , vol. 40, no. 10, p. 218, 2016.[7] A. Azaria, A. Ekblaw, T. Vieira, and A. Lippman, “Medrec: Usingblockchain for medical data access and permission management,” in
Open and Big Data (OBD), International Conference on . IEEE, 2016,pp. 25–30.[8] E. Kokoris-Kogias, P. Jovanovic, L. Gasser, N. Gailly, and B. Ford,“Omniledger: A secure, scale-out, decentralized ledger.”
IACR Cryptol-ogy ePrint Archive , vol. 2017, p. 406, 2017.[9] M. Zamani, M. Movahedi, and M. Raykova, “Rapidchain: A fastblockchain protocol via full sharding.”[10] G. O. Karame, E. Androulaki, M. Roeschlin, A. Gervais, and S. ˇCapkun,“Misbehavior in bitcoin: A study of double-spending and accountabil-ity,”
ACM Transactions on Information and System Security (TISSEC) ,vol. 18, no. 1, p. 2, 2015.[11] D. Kondor, M. P´osfai, I. Csabai, and G. Vattay, “Do the rich get richer?an empirical analysis of the bitcoin transaction network,”
PloS one ,vol. 9, no. 2, p. e86197, 2014.[12] D. George and S. Meiklejohn, “Centrally banked cryptocurrencies,” in
Network and Distributed System Security Symposium 2016 , 2015, pp.1–14.[13] L. Luu, V. Narayanan, C. Zheng, K. Baweja, S. Gilbert, and P. Saxena,“A secure sharding protocol for open blockchains,” in
Proceedings ofthe 2016 ACM SIGSAC Conference on Computer and CommunicationsSecurity . ACM, 2016, pp. 17–30.[14] M. Al-Bassam, A. Sonnino, S. Bano, D. Hrycyszyn, and G. Danezis,“Chainspace: A sharded smart contracts platform,” in , 2018.[15] “Sharding introduction r&d compendium,”https://github.com/ethereum/wiki/wiki/Sharding-introduction-R&D-compendium, 2018, accessed: 2019-01-12.[16] “Monoxide: Scale out blockchain with asynchronized consensus zones,”in . Boston, MA: USENIX Association, 2019.[17] I. Stanton and G. Kliot, “Streaming graph partitioning for large dis-tributed graphs,” in
Proceedings of the 18th ACM SIGKDD internationalconference on Knowledge discovery and data mining . ACM, 2012, pp.1222–1230.[18] Z. Abbas, V. Kalavri, P. Carbone, and V. Vlassov, “Streaming graph par-titioning: an experimental study,”
Proceedings of the VLDB Endowment ,vol. 11, no. 11, pp. 1590–1603, 2018.[19] G. Karypis and V. Kumar, “Metis–unstructured graph partitioning andsparse matrix ordering system, version 2.0,” 1995.[20] “Bitcoin network dataset,” https://senseable2015-6.mit.edu/bitcoin/,2018, accessed: 2019-01-12.[21] J. Pearson, “Wikileaks is now a target in themassive spam attack on bitcoin,” Jul 2015. [On-line]. Available: https://motherboard.vice.com/en us/article/ezvw7z/wikileaks-is-now-a-target-in-the-massive-spam-attack-on-bitcoin[22] I. Baumgart, B. Heep, and S. Krause, “OverSim: A flexible overlaynetwork simulation framework,” in
Proceedings of 10th IEEE GlobalInternet Symposium (GI ’07) in conjunction with IEEE INFOCOM 2007,Anchorage, AK, USA , May 2007, pp. 79–84.[23] A. Varga and R. Hornig, “An overview of the omnet++ simulationenvironment,” in