Building Blocks of Sharding Blockchain Systems: Concepts, Approaches, and Open Problems
Yizhong Liu, Jianwei Liu, Marcos Antonio Vaz Salles, Zongyang Zhang, Tong Li, Bin Hu, Fritz Henglein, Rongxing Lu
BBuilding Blocks of Sharding Blockchain Systems: Concepts, Approaches,and Open Problems
Yizhong Liu a , Jianwei Liu a , Marcos Antonio Vaz Salles b , Zongyang Zhang a, ∗ , Tong Li a , Bin Hu a ,Fritz Henglein b , Rongxing Lu c a School of Cyber Science and Technology, Beihang University, Beijing, China b Department of Computer Science, University of Copenhagen, Copenhagen, Denmark c Faculty of Computer Science, University of New Brunswick, Fredericton, Canada
Abstract
Sharding is the prevalent approach to breaking the trilemma of simultaneously achieving decentralization,security, and scalability in traditional blockchain systems, which are implemented as replicated state ma-chines relying on atomic broadcast for consensus on an immutable chain of valid transactions. Sharding is tobe understood broadly as techniques for dynamically partitioning nodes in a blockchain system into subsets(shards) that perform storage, communication, and computation tasks without fine-grained synchronizationwith each other. Despite much recent research on sharding blockchains, much remains to be explored inthe design space of these systems. Towards that aim, we conduct a systematic analysis of existing shardingblockchain systems and derive a conceptual decomposition of their architecture into functional componentsand the underlying assumptions about system models and attackers they are built on. The functional compo-nents identified are node selection, epoch randomness, node assignment, intra-shard consensus, cross-shardtransaction processing, shard reconfiguration, and motivation mechanism. We describe interfaces, function-ality, and properties of each component and show how they compose into a sharding blockchain system.For each component, we systematically review existing approaches, identify potential and open problems,and propose future research directions. We focus on potential security attacks and performance problems,including system throughput and latency concerns such as confirmation delays. We believe our modulararchitectural decomposition and in-depth analysis of each component, based on a comprehensive literaturestudy, provides a systematic basis for conceptualizing state-of-the-art sharding blockchain systems, provingor improving security and performance properties of components, and developing new sharding blockchainsystem designs.
Keywords:
Sharding Blockchain, Byzantine Fault Tolerance, Scalability, Throughput, Consensus,Modular Decomposition
1. Introduction
A traditional blockchain system is a peer-to-peer (P2P) distributed system with decentralized governance.It implements a replicated state machine that relies on an atomic broadcast (total event order consensus)protocol for consensus on an immutable chain (sequence) of valid transactions. A transaction can be anyrecord of data but is usually a digitally signed statement that expresses a transfer of ownership of a digitalresource (asset). In the seminal and paradigmatic Bitcoin system [1] , a transaction is a cryptographically ∗ Corresponding author
Email addresses: [email protected] (Yizhong Liu), [email protected] (Jianwei Liu), [email protected] (Marcos Antonio Vaz Salles), [email protected] (Zongyang Zhang), [email protected] (Tong Li), [email protected] (Bin Hu), [email protected] (Fritz Henglein), [email protected] (Rongxing Lu) Where a blockchain is called chain of blocks ; neither “blockchain” nor “block-chain” occur in the paper.
Preprint submitted to March 1, 2021 a r X i v : . [ c s . CR ] F e b igned transfer of a built-in synthetic currency, Bitcoin, by and to anonymous parties identified only bypublic keys they themselves have generated. Any node can join and leave the P2P network at any time,without authentication. The nodes receive transactions submitted by clients, collect them into blocks ofvalid transactions, and compete with other nodes to have their block extend the currently longest chain ofsuch blocks. The node operators are incentivized to do this by receiving a fee, in Bitcoin, whenever theirblock is the consensual continuation to the currently longest chain. An idealized blockchain system strives for the combination of decentralized control (no single party hasan a priori privileged role), consensus on a single state (“single source of truth”), tamper-proof recording(validated transactions cannot feasibly be deleted or updated ex post ), once added to the blockchain),privacy preservation (publicly shared data, but secured and privatized by cryptographic techniques such ascryptographic hashing, private-public key cryptography, secret sharing for digital signatures [2, 3], immutableself-certifying pointers [4], zero-knowledge proofs and more), and high availability (high degree of replicationon nodes controlled by independent, non-colluding node operators) in an untrusted environment [5].Blockchain systems have tremendous application potential in various fields, such as Internet of Things(IoT) [6, 7], cloud computing [8, 9] (with a trusted data center provider), smart cities [10], finance [11, 12, 13](with authenticated legal entities), self-sovereign identity management , supply chain [14] and more.Blockchain technology develops very rapidly, but faces both fundamental and practical obstacles towider applicability. The most critical issue is the trilemma of decentralization, security, and scalability. Toachieve decentralization, solutions need to support independent participants with varying assumptions onhow participants may join or leave a network, from permissioned to permissionless blockchain systems [15].As for security [16, 17], a blockchain protocol has to be proved secure [18, 19] to ensure it resists certainclasses of attacks that may compromise its correct functioning vis a vis client.For improved scalability, existing approaches can be classified into off-chain and on-chain solutions[20]. The former employ a hierarchical architecture, where the core blockchain system (on-chain) onlyvalidates few aggregated resource transfers that are the net effect of many fine-grained payments, whichin turn are managed separately in multiple unsynchronized subsystems. This includes technologies such asmicropayment [21, 22], payment channel networks [23, 24], virtual payment channels [25], and sidechains[26, 27, 28]. Conceptually, these solutions correspond to a (full-reserve) banking system, where the centralbank corresponds to the core blockchain system, and the channels to pop-up banks that have initially someaccounts transferred from the central bank, perform internal transactions without synchronization with otherbanks, and eventually transfer the final balances back to the central bank such that only the netted result isstored in the blockchain. Off-chain solutions avoid the computational cost of a traditional blockchain system,which requires each honest node to receive, store and send all transactions and to come to an agreementamongst all nodes on a total order of all valid transactions.Despite the applicability of off-chain solutions in some scenarios, on-chain solutions where all transactionsare recorded, validated, and retained without netting, are desired in many other application scenarios. A sharding blockchain system is an on-chain solution that seeks to improve the scalability of a traditionalblockchain system while achieving the same level of security and decentralization. In a sharding blockchainsystem, the nodes in the network are dynamically partitioned into shards (subsets), where each shard isresponsible for managing its own blockchain. The basic idea is that, instead of storing a chain of transactionsand replicating it across all nodes, an acyclic graph of transactions is maintained, where each shard is onlyresponsible for a specific part of the graph. As new nodes join the network, the cumulative transactionthroughput can grow by increasing the number of shards [29]. Sharding technology was pioneered fordatabase systems [30], where it describes methods for dynamically partitioning a database into parts, calledshards, each managed by a different node in a distributed system. The concept of partitioning data andtheir management, including the term sharding, was introduced to blockchain systems by ELASTICO [31].The design and implementation of sharding blockchain systems currently suffer from a number of prob-lems, however. First, the structure of sharding blockchain systems is complicated; they usually contain This is a simplified description with technical infelicities. Such as ESSIF, part of the European Blockchain Services Initiative, Sovrin or a number of Hyperledger identity managementframeworks. such as the method for selecting shard members, the consensus algorithm insidea shard, and the processing method for transactions. Second, different sharding blockchain systems mayadopt different models, such as network, adversary, and transaction models, without making these explicitenough to assess their design and (security) properties. These model assumptions engender a considerableset of design choices, which need to be characterized and compared in the context of their model assump-tions. Third, most sharding blockchain systems are presented as end-to-end systems, describing a specificpoint in a tremendous design space for blockchain systems, without providing an architecture for exploringsystematic modular design to rapidly and securely explore the design space for sharding blockchain systems.In particular, the functionality of components and their interfaces are not clear enough, which leads todifficulties in exploring various alternative designs for each component.In this paper, we address these points: We provide a conceptual and technical framework for decom-posing existing sharding blockchain systems into key functional components and describe a conceptual andtechnical modular architecture for composing them into full sharding blockchain systems. Besides, we pro-pose a taxonomy for sharding blockchain systems from the dimensions of system models and components.Furthermore, we provide a systematic, in-depth analysis of each component, describing its input dependen-cies, functionality, and key properties. For each component, we classify existing approaches and solutions,identify open problems, and provide directions for future research. In summary, we make the following contributions in this paper.
Decomposing sharding blockchains into functional components.
We decompose sharding block-chains into multiple functional components: node selection, epoch randomness, node assignment, intra-shardconsensus, cross-shard transaction processing, shard reconfiguration, and motivation mechanism. For eachcomponent, we give its input, output, function, and property to be satisfied. Furthermore, we show howto compose these components into a complete sharding blockchain system. The component decompositionprovides a path to systematically developing yet unexplored sharding blockchain system designs.
A detailed taxonomy for sharding blockchain systems.
We provide a taxonomy for sharding block-chain systems in two dimensions: system models and components. System models include network model,adversary model, and transaction model. We divide each model into categories that correspond to differenttypes of sharding blockchain systems, such as eventual and instant sharding blockchain systems, or per-missioned and permissionless sharding blockchain systems. For each component of a sharding blockchainsystem, we classify solutions according to their principles and algorithms.
In-depth analysis of components . For each component of a sharding blockchain system, we first give itsbasic concepts, including its purpose, functionality, and essential procedures. We summarize and categorizeexisting approaches according to their underlying characteristics; the specific operational details of eachmethod are expounded from multiple perspectives. In addition, we identify and analyze possible problemsfor every type of solution, including attacks an adversary might launch on security, throughput, and la-tency (transaction confirmation delays). Finally, we point out possible future research directions for eachcomponent.
Section 2 gives the background, definitions and notations that are useful in this paper. In Section 3we decompose sharding blockchains into several components and discuss methodologies to derive shardingblockchain systems from their composition. In Section 4, Section 5, and Section 6, basic concepts, existingapproaches, open problems and future directions of node selection, epoch randomness, and node assignmentare given, respectively. These three parts involve the method to confirm shard members. Then Section 7describes the classic state machine replication algorithms that are used as intra-shard consensus. In Sec-tion 8, cross-shard transaction processing methods are analyzed in detail, which is important to all shardingblockchain systems. Section 9 and Section 10 give the existing approaches and potential problems about We use the terms “functional component” and “building block” interchangeably.
2. Preliminaries
In this section, we first give the background of sharding blockchains. Then notations and definitions thatare useful in the paper are described in detail.
Sharding blockchains adopt a unique blockchain consensus mechanism. Next, we first describe therelevant background of blockchain consensus and then introduce sharding blockchains.
Blockchain technology is introduced by Bitcoin [1] in 2008, which realizes the agreement of the ledgeramong distributed nodes through a specific consensus mechanism. The reason why the blockchain is called“chain” is because each block is linked to the previous one in a specific cryptographic way. The contentsstored in a block mainly include the transactions generated in the network during each period of time.Blockchain technology is currently a hot area of research and has great application potential since it hasthe following characteristics. The first characteristic is decentralization. Decentralization means that thereis no trusted third party in the network, which is different from the traditional centralized transaction mode.The second characteristic is trustlessness, which means that nodes do not need to trust each other, and canfinally reach consensus on the ledger through a specific consensus mechanism. The third characteristic istransparency. In a permissionless network (ref. Definition 6), all nodes can join and leave the protocol at anytime, and nodes could obtain the historical ledger data of the blockchain at any time. The fourth is tamper-resistance. Under the assumptions of appropriate network and adversary models, historical data in theblockchain cannot be illegally tampered with, except for some special redactable blockchains [32, 33]. Oncea block is confirmed (strong consistency blockchains, ref. Definition 20), or a block reaches a certain depth(weak consistency blockchains, ref. Definition 16), the contents of the block can no longer be modified. Thefifth is anonymity. Through some approaches (such as transaction graph analysis [34, 35]), deanonymizationanalysis on historical transaction data could be performed on some blockchains. However, by adopting somecryptographic technologies, such as blind signatures [36, 37], ring signatures [38, 39], and zero-knowledgeproofs [40, 41], privacy-protection blockchains are designed to prevent users’ privacy from leakage.As shown in Fig. 1, the architecture of blockchain could be divided into several layers, including anetwork layer, a consensus layer, and an application layer. In the network layer, participating nodes join aP2P communication network [42] to synchronize information with each other. In a P2P network, the nodesare distributed, and there is no central communication node in the network. The transfer and update ofinformation are completed by the P2P communication between each node. Besides, there could be otherkinds of connecting nodes in a blockchain network, such as databases, IoT devices, and lightweight clients.In the consensus layer, the participating nodes in the network with certain computing and communicationcapabilities act as consensus nodes, that is, block producers (we use the term “block producer” instead of“miner” to make the meaning more general), and generate blocks through certain consensus algorithms.Note that there are many types of consensus algorithms, and Fig. 1 shows a Byzantine Fault Tolerance(BFT) [43] algorithm. Many applications, such as digital assets [44], smart contracts [45], and decentralizedapplications [46], could be built based on a blockchain, together constituting the application layer.A consensus mechanism is very critical to a blockchain system, and to a large extent determines thesecurity and performance of the system. A blockchain consensus is to ensure the consistency and liveness ofthe system by formulating rules for nodes to participate in the blockchain protocol. Consistency means thatall honest nodes ultimately have the same view of the blockchain. Liveness refers to that the transactionsuploaded by users to the blockchain are sure to be processed after a certain period of time.At present, the blockchain consensus mechanisms could be divided into the following categories, ac-cording to [47]. First, it is the classic distributed consensus algorithm, represented by the Byzantine fault4 igure 1: Blockchain Layers. tolerance algorithms, which implements state machine replication (ref. Definition 21) in a limited number ofparticipating nodes. The second is proof-of-work (PoW) based consensus, including Bitcoin [1], Bitcoin-NG[48], GHOST [50], FruitChains [49], SPECTURE [51], etc. Besides, there is proof-of-stake (PoS) based con-sensus, such as Casper FFG [52], Snow White [53], Ouroboros [54], etc. Finally, the hybrid consensus, suchas PeerCensus [55], ByzCoin [56], Solida [57], etc., combines classic state machine replication algorithms andblockchain technology. According to the number of committees, the hybrid consensus mechanisms could bedivided into single-committee and multi-committee hybrid consensus mechanisms. The multi-committeehybrid consensus is a kind of sharding consensus mechanism, which is introduced in the following.
Sharding technology is first proposed and used in the field of databases [30]. By dividing all participatingnodes in the network into multiple shards, each shard is only responsible for maintaining its own correspond-ing data. In this way, the scalability of network processing capabilities could be achieved. As the numberof nodes in the network increases, the enhancement of processing capabilities is realized by adding moreshards. The sharding blockchain is first proposed by ELASTICO [31], which combines sharding technologyand blockchain technology, with the purpose to increase the transaction throughput, i.e., the number oftransactions processed per second. Since then, there have been many studies on sharding blockchains, suchas Omniledger [58], Chainspace [58], RapidChain [59], and Monoxide [60].In general, sharding blockchains have the following three characteristics. The first one is communicationsharding. Participating nodes are divided into different shards where nodes in each shard only need internalcommunication most of the time. The clients and nodes within each shard could obtain the current stateof the blockchain by communicating with the intra-shard nodes that are responsible for maintaining theblockchain, e.g., a committee. The second one is computation sharding, which means that each shard isonly responsible for processing its corresponding transactions. The distribution of transactions to shards isdiversified, e.g., by selecting the corresponding shard according to the transaction ID. Generally speaking,according to the last several bits of the transaction ID, it is determined which shard the transaction belongsto. So transactions are handled by different shards in parallel. When the number of nodes in the networkincreases, more shards could be added to realize scalability. The third one is storage sharding. Storagesharding means that nodes of different shards only needs to store the data of its corresponding shard.The data includes transaction history and unspent transaction output (UTXO, ref. Definition 14) data.Transaction history data exists in the form of a blockchain, while the UTXO data could be derived from5ransaction history data or could be stored separately to improve its processing efficiency. Storage shardingallows nodes to store a fraction of the entire blockchain system data, reducing the storage burden of nodes.As shown in Fig. 2, communications play an important role in sharding blockchain systems. Nodes in thesame shard only need to execute intra-shard communication most of the time and send some key informationto a coordinator of the shard. The coordinator is usually responsible for cross-shard communication as wellas intra-shard consensus, and each shard has at least one coordinator. Generally speaking, the coordinatorneeds to have stronger communication capabilities than other intra-shard nodes.
Figure 2: Communications in Sharding Blockchain Systems.
Note that some methods, such as OHIE [61] and Prism [62], are designed to improve the throughputof the blockchains, while in a strict sense, sharding technology is not used in their structures. Hence, wewill not introduce this type of solutions, while focusing on the blockchains that use sharding technology. Asharding blockchain usually includes several major components such as node selection, epoch randomness,node assignment, intra-shard consensus, cross-shard transaction processing, and shard reconfiguration. Eachcomponent could be implemented by different methods, and then by composing all the components, acomplete sharding blockchain could be obtained. We will give a detailed analysis of each component andtheir composition approach in this paper.
We summarize the notations that are used in our paper in Table 1. To make it compatible with otherprotocols, we use 𝑛 to denote the total number of participating nodes, while use 𝑢 to denote the numberof members in a committee. So inside a committee, the security condition that should be satisfied couldbe represented by 𝑢 = 𝑓 + 𝑢 = 𝑓 +
1, where 𝑓 is the number of admissible failures. In addition, shard 𝑖 denotes the 𝑖 -th shard, while the notation C 𝑖 denotes the committee in shard 𝑖 . These two conceptsare different since there might be no committee in a shard in some kinds of sharding blockchains. Thenotation LOG means the output log, or ledger of a shard, while chain denotes a blockchain. The specificmeaning of
LOG and chain are similar, while there are also differences. A blockchain chain might containother contents and information, such as block headers and additional information in OP RETURN [63].
LOG usually means extracting key information from a blockchain, e.g., transaction data.6 able 1: Notations
Symbol Definition tx a transaction∆ the upper bound of the network’s delay 𝛿 the actual delay of the network 𝑢 the exact number of members in a shard committee ★ 𝑓 the maximum number of malicious node in a shard committee 𝑚 the total number of shards 𝑛 the total number of nodes that participate in the protocol shard 𝑖 the 𝑖 -th shard C 𝑖𝑒 the 𝑖 -th committee of epoch 𝑒 * C 𝑅𝑒 the reference committee of epoch 𝑒 LOG 𝑖 the output log of the 𝑖 -th shard’s nodes chain a blockchain 𝜌 the fraction of the computational power that is held by an adversary 𝜏 the corruption time parameter of an adversary 𝑝 the probability that one node finds a PoW solution successfully in one single round ★ For the convenience of analysis, we assume that the number of members in a committee is fixed. * The concept of shard and committee is different. In committee-based sharding blockchains, there is acommittee responsible for processing transactions in each shard, and we call it an ordinary committee.So the number of ordinary committees is also 𝑚 . Besides, some sharding blockchains, e.g., RapidChain[59], utilize a reference committee to confirm committee members. To make our description clearer, we give the definitions that are useful in our paper and helpful in un-derstanding sharding blockchains. As shown in Fig. 3, we divide the definitions into several major categoriesand give their relationships. Each category of definitions could be seen as a tree, where we can select oneof the leaf nodes in each tree to form a sharding blockchain. In the following, we introduce these defini-tions related to network model, adversary model, transaction model, intra-shard consensus, and shardingblockchains.
To describe the network model more precisely, we divide network models into message transmissionmodels and node admission models as follows. Message transmission model.
We assume that nodes participate in the network with authentication. Themessages sent by the nodes are signed, and an adversary cannot forge the signature of any honest node.There are different models for the message delay rules. Next, we give the definitions of three differentmessage transmission models.
Definition 1 (Synchronous Network [66]).
In a synchronous network, messages between honest nodesare propagated in rounds. In each round, the messages sent by honest nodes can reach all other honest usersbefore the next round. Each round has a fixed length of time. Note that in other work [64, 65], network model mainly refers to message transmission model in our paper. igure 3: Definitions and their relationships. Synchronous networks are relatively strong network models.There are two kinds of definitions for a partially synchronous network.
Definition 2 (Partially Synchronous Network-Definition A[66]).
In a partially synchronous network,there is a certain upper bound Δ of message transmission delay in the network. The parameter Δ cannot beused as a parameter in the design of a protocol. Definition 3 (Partially Synchronous Network-Definition B [67]).
In a partially synchronous net-work, there is a known bound Δ and an unknown Global Stabilization Time (GST), such that after GST, alltransmissions between two honest nodes arrive within time Δ . Under this definition, a protocol usually ensures safety and guarantees progress within a bounded dura-tion after the GST. The partially synchronous network is a commonly used network model in the analysisof blockchain protocols.
Definition 4 (Asynchronous Network [67]).
In an asynchronous network, there exists an adversarywho can arbitrarily delay or reorder messages of honest nodes, as long as the messages of honest users canreach each other. There is no upper limit of message transmission delay.
About the asynchronous network, the FLP impossibility theorem [68] argues that in an asynchronoussystem where the network is reliable but where crash failures are allowed, there is no deterministic consensusmechanism that can solve the consensus problem. 8ost blockchain protocols adopt one of the above three models. However, in a sharding blockchain, twosituations need to be considered. The first is the model of the entire network, and the second is the internalmodel of each shard. These two models could be the same or different and need to be analyzed accordingto actual conditions and requirements.
Node admission model.
Note that we assume that the nodes in the network are homogeneous, i.e., eachnode has close computation and communication capabilities. In blockchain protocols, the rules for nodes toparticipate in the protocols are different. We name these rules as node admission models and divide theminto permissioned and permissionless networks as follows.
Definition 5 (Permissioned Network).
A permissioned network means that the nodes participating inthe protocol must first complete identity authentication.
Identity authentication is usually done through a trusted third party, e.g., a certificate authority (CA).In a permissioned network, the number and identities of all nodes are known. This model is mostly adoptedby some internal protocols of enterprises or units [69].
Definition 6 (Permissionless Network).
In a permissionless network: • Any node can join or leave at any time; • No identity authentication is required; • The number of participating nodes varies at any time and is unpredictable.
So in a permissionless network, information about the number and identity of all participant nodesis unknown. The read and write rights of the data are generally open to all nodes, guaranteeing thedecentralization property [29].The words “permissioned” and “permissionless” in the above two models can usually be combined withdifferent terms, such as “permissioned blockchain”, which means a blockchain protocol in a permissionednetwork, or “permissionless consensus”, which means a consensus algorithm in a permissionless network.
The adversary model describes the various capabilities of an adversary in a protocol. We divide theadversary model into the corruption model, the total proportion model, and the intra-shard proportionmodel.
Corruption model.
In this model, an adversary could completely control a target node and obtain its secretinformation, the input and output messages.We describe an adversary’s corruption ability from two aspects, i.e., corruption timing and corruptionspeed. First, according to the timing at which an adversary can launch a corruption attack, the corruptionmodel could be divided into static and adaptive corruption. Second, according to the time taken by anadversary to complete the corruption attack, there are mild corruption and immediate corruption.
Definition 7 (Static Corruption).
A static corruption means that an adversary can only select its cor-ruption targets before the protocol starts. Once the protocol starts running, it cannot choose other honestnodes to corrupt.
Definition 8 (Adaptive Corruption [70]).
Adaptive corruption means that an adversary is able to dy-namically corrupt target nodes according to the information collected during the operation of the protocol.
The above two models are usually considered in some cryptographic protocols. In the context of shardingblockchains, it is more important to consider the corruption speed since this will affect the reconfigurationprocess of a sharding blockchain. Specifically, if the shard members remain unchanged, then an adversarymay corrupt and control one of the shards after a period of time. Therefore, a sharding blockchain needs toupdate committee members at regular intervals, where the interval is called an epoch. In order to make thedescription clearer, we introduce the definition of epoch here.9 efinition 9 (Epoch).
An epoch in a sharding blockchain refers to the time period during which all shardmembers remain unchanged and continue to operate. Different epochs correspond to different shard memberconfigurations.
A reconfiguration refers to switching from one epoch to another, i.e., the process of updating the shardmembers. The time length of an epoch is closely related to the time required for the adversary to completethe corruption. To better describe this, we give the following two definitions.
Definition 10 (Mild Corruption [71]).
Mild corruption means that it takes a certain amount of timewhich is usually denoted as 𝜏 for an adversary to corrupt a node. An adversary first issues the corruptioncommand at time 𝑡 to the target node. After 𝜏 time, the target node is corrupted and becomes a maliciousnode. Before time 𝑡 + 𝜏 , the target node remains honest. We call 𝜏 the corruption time parameter in this paper. Definition 11 (Immediate Corruption).
Immediate corruption means that an adversary’s corruptionattack is effective immediately, i.e., 𝜏 = . At present, most sharding blockchains adopt the mild corruption model. As far as we know, the imme-diate corruption model is only used in few blockchain protocols, e.g., Algorand [72].
Total proportion model.
The total proportion model refers to a certain limit on the proportion of computa-tional power or stake that an adversary can control in the whole network. For an entire blockchain protocol,a percentage or fraction is generally used. The common used total proportion model might be denoted by25%, 1/3, 49%, etc. The adversary proportion is less than or equal to these specific percentages or fractions.For the sake of consistency, we use fractions to indicate the total proportion of the adversary in thispaper. According to its relationship with the intra-shard proportion, we divide the total proportion into [ , / ) and [ / , / ) , as shown in Fig 3. [ , / ) means that the proportion is greater than or equal to 0and less than 1 /
3. Similarly, [ / , / ) refers to the proportion greater than or equal to 1 / / Intra-shard proportion model.
Generally speaking, the representation of an adversary model in a shardis different, since the number of shard members is usually limited and fixed. Specifically, the relationshipbetween the number of nodes controlled by an adversary and the total number of shard members is expressedin the form of an equation. 𝑓 is used to represent the number of malicious nodes, and 𝑢 denotes the totalnumber of nodes in a shard. The intra-shard adversary proportion model could be described as follows.
Definition 12 ( 𝑢 = 𝑓 + ). The proportion of malicious nodes that an adversary controls accounts for nomore than 1/2 of the whole shard.
Definition 13 ( 𝑢 = 𝑓 + ). The proportion of malicious nodes that an adversary controls accounts for nomore than 1/3 of the whole shard.
These two models are usually utilized when there are committees running some BFT algorithms in theshards. 𝑢 = 𝑓 + 𝑢 = 𝑓 + In other papers, 𝑛 is usually used to denote the number of shard members. In this paper, we use 𝑛 to represent the totalnumber of nodes in the protocol. To avoid conflicts, we use 𝑢 to denote the number of members in a shard. .3.3. Transaction Model The main function of most blockchain systems is to process transactions. A transaction usually containsinformation such as timestamp, input, output, signature, etc., and is often used to realize the transfer ofproperty. Different blockchain systems may adopt different transaction models. The existing transactionmodels are mainly divided into the UTXO model, account model, and others. The UTXO model and accountmodel are the most commonly used models, and we term such a model a “generic” one. Others refer tosome special transaction models. For instance, in Hyperledger Fabric [69], one can create transactions bynot associating a balance with an account.
UTXO model.
The UTXO model is the most commonly used blockchain transaction model.
Definition 14 (UTXO Model [73]).
In the UTXO model, assets (money/coins/stakes) are stored in un-spent transaction outputs (UTXOs). Each UTXO contains the public key address of the output and its value.Each transaction spends existing UTXOs and creates new UTXOs, essentially transferring assets from theinput public key address to the output public key address. Besides, a valid transaction requires that the sumof values in the output UTXOs must be equal to that in the input UTXOs.Account model.
The account model is another common blockchain transaction model defined as follows.
Definition 15 (Account Model [73]).
In the account model, each user has a fixed account. The accountinformation includes the account address and account balance, and the account balance must be non-negative.A transaction is to transfer assets from one account to another account. A valid transaction requires thatthe balance in the input account is greater than or equal to the transaction amount.2.3.4. Intra-Shard Consensus
Intra-shard consensus is crucial to sharding blockchains. Next, we will introduce definitions related tointra-shard consensus in terms of weak and strong consistency.
Weak consistency.
Weak consistency is also known as eventual consistency, which is defined as follows.
Definition 16 (Weak Consistency).
Weak consistency means that different nodes might end up havingdifferent views of a blockchain, which leads to forks. A certain number of blocks at the end of the blockchainneed to be truncated to obtain stable transactions.
Specifically, based on the election method of a block producer, there might be two or more legal blockproducers in the same round. In this case, a short-term fork might appear in a blockchain. However, after acertain period of time, a final blockchain is determined according to some kind of decision rule, such as thelongest chain rule of Bitcoin [1] or the heaviest chain rule of GHOST [50]. Other blockchain systems thathave weak consistency property are PPCoin [74], Ouroboros [54], etc.The following three properties proposed by the Bitcoin backbone protocol [18] are suitable for blockchainswith weak consistency property.
Definition 17 (Common Prefix [18]).
For any two blockchains chain , chain output by any two honestnodes 𝑃 , 𝑃 in any two rounds 𝑟 , 𝑟 , it holds that chain (cid:100) 𝑘 (cid:22) chain (cid:100) 𝑘 or chain (cid:100) 𝑘 (cid:22) chain (cid:100) 𝑘 where 𝑘 ∈ N is the security parameter. That is to say, when 𝑟 ≤ 𝑟 is satisfied, it holds that chain (cid:100) 𝑘 (cid:22) chain (cid:100) 𝑘 ; when 𝑟 ≤ 𝑟 is satisfied, it holds that chain (cid:100) 𝑘 (cid:22) chain (cid:100) 𝑘 . chain (cid:100) 𝑘 means removing the ending 𝑘 blocks of chain , chain (cid:22) chain denotes that chain is a prefix of chain . 𝑃 , 𝑃 might be the same node. The common prefix property could be understood in the following way. The blockchains held by honestnodes will eventually be consistent with each other, and the stable part of a blockchain will not be rewritten.After removing the last 𝑘 blocks, the remaining blockchain is regarded as the stable part. The value of 𝑘 isusually related to system security. 11 efinition 18 (Chain Quality [18]). For any blockchain chain output by any honest node 𝑃 , after re-moving the latest 𝑘 blocks, the proportion of malicious blocks in any 𝑘 consecutive blocks does not exceed 𝜇 . The security parameters are 𝜇 ∈ R , 𝑘, 𝑘 ∈ N . The chain quality property means that there must be a sufficient proportion of consecutive blocks gen-erated by honest users. The last 𝑘 blocks are usually the “unstable” blocks at the end of a blockchain. Definition 19 (Chain Growth [18]).
For any round 𝑟 ( 𝑟 > 𝑟 ) and any honest node 𝑃 , if 𝑃 outputs chain and chain in round 𝑟 and 𝑟 + 𝑠 , respectively, then it holds that | chain | − | chain | ≥ 𝜏 · 𝑠 . The securityparameters are 𝜏 ∈ R , 𝑠, 𝑟 ∈ N . The chain growth property means that a blockchain will continuously generate new blocks, and the blockgeneration speed has a lower bound.
Strong consistency.
Strong consistency means that there is no need to wait for a block to reach a certaindepth to confirm transactions. Blocks and transactions are confirmed immediately.
Definition 20 (Strong Consistency).
Strong consistency means that the generation of each block is de-terministic and instant. Besides, strong consistency has the following characteristics: • There is no fork in a blockchain. By running a distributed consensus algorithm, state machine repli-cation (ref. Definition 21) is achieved; • Transactions could be confirmed more quickly. As long as a transaction is written into a block, thetransaction could be regarded as valid; • Transactions are tamper-proof. As long as a transaction block is written to a blockchain, the transactionand block will not be tampered with and the block will remain on the chain at all times.
The blockchain systems that have strong consistency property are PeerCensus [55], ByzCoin [56] and itsadaptation MOTOR [75], Hybrid Consensus [76], Solida [57], Omniledger [58], etc. In these blockchains,there is a committee or multiple committees that run distributed consensus algorithms, e.g., PBFT [43], toconfirm transactions and generate new blocks. These distributed consensus algorithms achieve state machinereplication, which is defined as follows.
Definition 21 (State Machine Replication [77]).
State machine replication is a general method for aset of servers, which include a single primary and other backups, to reach an agreement on a linearly-orderedlog, where the following two security properties must be satisfied. • Consistency, i.e., the views of all honest servers must be identical to each other. • Liveness, i.e., whenever one piece of valid data is submitted to the servers, it will be written to the logwithin some bounded time.
State machine replication is also known as atomic broadcast, and it has been studied for decades in thearea of distributed systems. State machine replication is often used to synchronize large databases. Forexample, Google and Facebook use it for the synchronization of core parts of their databases [78].State machine replication needs to tolerate a specific proportion of faulty nodes. A faulty node mightbe a crashed node or a Byzantine node. A crashed node refers to a node that does not respond. The nodemay have a system failure or be offline. A crashed node is a relatively simple faulty node compared with aByzantine node defined as follows.
Definition 22 (Byzantine Node).
A node is called a Byzantine node if it could behave arbitrarily asfollows [79]. • It does not respond to messages sent to it; It sends different messages to different nodes when such messages were supposed to be identical.Byzantine nodes are considered to be controlled by an adversary. A Byzantine node must observe some re-strictions, which are also restrictions on the adversary, usually given in the network model and the adversarymodel.
The Byzantine node model is a commonly used fault model in distributed systems, and an algorithmthat can tolerate such nodes is called a Byzantine Fault Tolerance (BFT) algorithm, which is defined asfollows.
Definition 23 (Byzantine Fault Tolerance [79]).
A set of nodes achieve state machine replication andsatisfy consistency and liveness in the presence of Byzantine nodes.
There is usually a certain constraint on the proportion of Byzantine nodes. For instance, in a partiallysynchronous network model, the system is usually able to tolerate at most 1 / / After determining the network model, adversary model, transaction model, intra-shard consensus, andother components, a sharding blockchain is constructed. We give a formal definition of a secure shardingblockchain. Since this definition is based on that of a “public ledger” by Garay et al. [18], we first introducethe definition of a ledger.
Definition 24 (Ledger).
A ledger refers to a credible “bulletin board” that meets the following properties: • Persistence. Persistence means that for any honest node, if the node outputs a transaction tx at acertain position in his ledger, such as the 𝑗 -th transaction of the 𝑖 -th block, then tx must appear in theidentical position in the ledgers of all honest nodes. • Liveness. Liveness means that if a valid transaction tx is uploaded at time 𝑡 , then after a certain time, tx must appear in the ledgers output by all honest nodes.The participating nodes are able to write some data on a ledger if the data meets the specific rules of theledger. Note that in the original Bitcoin backbone protocol [18], the definition of a public ledger is given. In orderto make the concept of “ledger” better applicable to various network models, we make a slight modificationand directly define a ledger. Then different node admission models correspond to different types of ledgers,i.e., a private ledger in a permissioned network, and a public ledger in a permissionless network.
A secure sharding blockchain.
A sharding blockchain could be regarded as a special type of ledger. In orderto make the description clearer, we give a relatively formal definition of a secure sharding blockchain in thefollowing.
Definition 25 (A Secure Sharding Blockchain).
Let (A , Z) be an adversary and environment pairw.r.t. a sharding consensus protocol Π . 𝑇 initial denotes the time for a sharding blockchain protocol tostart up, including the production of genesis blocks and initial committees. 𝑇 liveness denotes the transactionconfirmation delay parameter, i.e., the time required to commit a transaction. We say Π is secure w.r.t. (A , Z) with parameters 𝑇 initial , 𝑇 liveness if the following properties hold with an overwhelming probability: • Consistency. Consistency includes the following two properties: – Common prefix inside a shard: For any two honest nodes 𝑖, 𝑗 ∈ shard 𝑐 where 𝑐 ∈ [ , 𝑚 ] , node 𝑖 outputs LOG 𝑖 to Z at time 𝑡 , and node 𝑗 outputs LOG 𝑗 to Z at time 𝑡 (cid:48) , it holds that either LOG 𝑖 (cid:22) LOG 𝑗 or LOG 𝑗 (cid:22) LOG 𝑖 . No conflict between shards: For any two honest nodes 𝑖 ∈ shard 𝑐 , 𝑗 ∈ shard 𝑐 (cid:48) where 𝑐, 𝑐 (cid:48) ∈ [ , 𝑚 ] and 𝑐 ≠ 𝑐 (cid:48) , node 𝑖 outputs LOG 𝑖 to Z at time 𝑡 , and node 𝑗 outputs LOG 𝑗 to Z at time 𝑡 (cid:48) . For anytransaction tx ∈ LOG 𝑖 and tx ∈ LOG 𝑗 where tx ≠ tx , it holds that tx and tx don’t conflict witheach other, i.e., there is no input that belongs to tx and tx simultaneously. • 𝑇 liveness -Liveness: For any honest node from any shard, if it receives a transaction tx at time 𝑡 ≥ 𝑇 initial from Z , then at time 𝑡 + 𝑇 liveness , tx must be accepted or rejected. Note that in the description of “no conflict between shards”, we require that the condition tx ≠ tx holdssince different shards might record the same cross-shard transaction in their blockchain, respectively. Theessence of “no conflict between shards” is that no double-spending happens for any UTXO. This paper usesthe term ”sharding blockchain” and ”secure sharding blockchain” interchangeably for the same meaning. Transaction confirmation.
Generally, the main purpose of a blockchain is to complete transaction processingand confirmation. In the following, the definition of transaction confirmation delay and responsiveness aregiven, both of which are very important evaluation indicators.
Definition 26 (Transaction Confirmation Delay).
For a transaction tx , if it is submitted by a clientat some time 𝑡 , and it appears in an honest node’s ledger at time 𝑡 (cid:48) , then 𝑡 (cid:48) − 𝑡 is the transaction confirmationdelay. 𝑡 (cid:48) − 𝑡 could also be regarded as the liveness parameter of a ledger. Transaction confirmation delay refers to the time needed for a valid transaction to be confirmed by ablockchain. Namely, it ranges from the time that the transaction is submitted by a client, to the time thatthe transaction appears in an honest node’s ledger.Besides, Pass and Shi [76] propose the concept of responsiveness.
Definition 27 (Responsiveness [76]).
Responsiveness means the confirmation time of a transaction isonly related to the actual network delay 𝛿 , but not to a priory upper bound Δ . The concept of responsiveness is used in much related research [80, 81] now as an evaluation indicatorfor transaction confirmation.The systematic introduction of these definitions aims at supporting the concept of composability ofcomponents in Section 3. Specifically, as Fig. 3 indicates, we could select one of the leaf nodes from each“tree” on the left to form a complete sharding blockchain system. For instance, in Fig. 3, we have selectedthe “Partially Sync” in the “Message Transmission Model”, the “PoW” and “Permissionless” in the “NodeAdmission Model”, the “Adaptive” in the “Corruption Timing”, the “Mild” in the “Corruption Speed”,the “UTXO Model” in the “Transaction Model”, and the “Byzantine Fault Tolerance” in the “Intra-ShardConsensus” trees. In this way, we obtain a complete sharding blockchain system that is very similar in itsassumptions to Omniledger [58].Interestingly, through this choice-combination approach, we could obtain a “sharding blockchain systemgenerator” which could be used to produce a variety of different sharding blockchain systems. Consideringthe practical combinations, the corruption timing and speed is usually set to “Adaptive” and “Mild”,respectively. For the node admission model, intra-shard consensus, and transaction model, if we exclude the“Others” leaf options, then there are 3 ∗ ∗ ∗ ∗ ∗ ∗ =
216 types of different secure sharding blockchainsystems that might be composed. The constraints between some models have been taken into consideration,such as the total proportion and the intra-shard proportion.
3. Decomposing Sharding Blockchains into Functional Components
In this section, we decompose sharding blockchains into functional components. Section 3.1 describes theinputs, outputs, basic functions, and properties of each component. Section 3.2 provides concrete methods tocompose different components into a complete sharding blockchain system. Section 3.3 summarizes existingsharding blockchain schemes. 14 .1. Decomposition of Sharding Blockchains
Inspired by the concept of a framework in [82], we propose a framework for sharding blockchains anddecompose it into functional components, including node selection, node assignment, epoch randomness,intra-shard consensus, cross-shard transaction processing, shard reconfiguration, and motivation mechanism.These components could be composed into a complete sharding blockchain system. The schematic diagramof a sharding blockchain is shown in Fig. 4.
Figure 4: The schematic diagram of a sharding blockchain.
Next, we give an informal description of each component. In each component, we describe the interfaces(inputs and outputs), basic functions, and properties to be satisfied.
As shown in Component 1, Ns is a subset selection component. Ns takes in 𝑝𝑛𝑜𝑑𝑒𝑠 as an input, whichrepresents a set of participating nodes in the network, and outputs 𝑠𝑛𝑜𝑑𝑒𝑠 , which denotes a set of selectednodes. { 𝑃 𝑖 } | 𝑘𝑚 | denotes that there are 𝑘𝑚 nodes in the set. 𝑠𝑛𝑜𝑑𝑒𝑠 is a subset of 𝑝𝑛𝑜𝑑𝑒𝑠 , and 𝑠𝑛𝑜𝑑𝑒𝑠 mightbe used by the Na component. 𝑘 denotes the number of old members that each shard needs to replace duringeach reconfiguration (equal to the number of newly added members of each shard). So 𝑘𝑚 represents the15umber of selected nodes required for each reconfiguration. In order to select 𝑘𝑚 nodes, the total numberof participating nodes 𝑛 must be greater than 𝑘𝑚 . Component 1: Ns (Node Selection) input: a set of participating nodes 𝑝𝑛𝑜𝑑𝑒𝑠 = { 𝑃 , · · · , 𝑃 𝑛 } . output: a subset of 𝑝𝑛𝑜𝑑𝑒𝑠 containing selected nodes 𝑠𝑛𝑜𝑑𝑒𝑠 = { 𝑃 𝑖 } | 𝑘𝑚 | . function: select a certain number of qualified nodes from all participating nodes. property: fairness; robustness.The basic function of Ns is to select qualified shard members from all participating nodes. In a permis-sionless network, each node could be a member of 𝑝𝑛𝑜𝑑𝑒𝑠 . Ns might make use of some mechanisms, such asPoW and PoS, to defend against the Sybil attacks [83], where an adversary tries to increase his proportionin 𝑠𝑛𝑜𝑑𝑒𝑠 by creating fake identities. In a permissioned network, the function of Ns is implemented bya trusted third party, e.g., CA. The CA completes the confirmation of 𝑠𝑛𝑜𝑑𝑒𝑠 by providing the identityregistration service.The properties that need to be satisfied by Ns are fairness and robustness. The definition of fairnessis given in Definition 31, which is mainly used to limit the proportion of the adversary’s nodes in 𝑠𝑛𝑜𝑑𝑒𝑠 .Robustness means that even with the participation of the adversary, 𝑠𝑛𝑜𝑑𝑒𝑠 will still be confirmed by honestnodes. As shown in Component 2, Er is usually an interactive and distributed component where each participantcreates a private input, namely 𝑥 , 𝑥 , · · · , 𝑥 𝑞 . We use 𝑞 to denote the number of participants. The epochrandomness is denoted by 𝜉 𝑒 where 𝑒 represents the epoch number. Component 2: Er (Epoch Randomness) input: 𝑞 private inputs 𝑥 , 𝑥 , · · · , 𝑥 𝑞 . output: an epoch randomness 𝜉 𝑒 . function: participants run a randomness generation protocol to generate a secure randomness. property: public-verifiability; unpredictability; bias-resistance; availability.The basic function of Er is to enable each honest node to obtain an identical randomness through theinteractions with participants. The randomness must be secure, i.e., satisfies the following properties: public-verifiability, unpredictability, bias-resistance, and availability. Public-verifiability means that every node canverify the correctness of 𝜉 𝑒 . Unpredictability indicates that no one can obtain the randomness in advance.Bias-resistance means that the adversary?s participation will not affect the result of the randomness andavailability is to ensure that the randomness is sure to be generated. The concrete explanation of theseproperties for a randomness is given in Section 5.1. An epoch randomness is usually utilized as a seed toassign nodes randomly into shards and used as a fresh puzzle in PoW mining. As shown in Component 3, Na represents the node assignment component, which takes in 𝑠𝑛𝑜𝑑𝑒 and 𝜉 𝑒 as inputs, and outputs 𝑎𝑛𝑜𝑑𝑒𝑠 . 𝑎𝑛𝑜𝑑𝑒𝑠 denotes an assigned node list which might contain 𝑚 groups ofnodes. { 𝑃 𝑖 } | 𝑘 | denotes that there are 𝑘 elements in the set. Na is to map the 𝑘𝑚 nodes in 𝑠𝑛𝑜𝑑𝑒𝑠 to a set 𝑎𝑛𝑜𝑑𝑒𝑠 that includes 𝑚 different subsets 𝑎 , · · · , 𝑎 𝑚 , and each subset 𝑎 𝑗 contains 𝑘 nodes.The basis function of Na is to assign the selected new nodes randomly to multiple shards. The randomdistribution of new nodes is required to prevent an adversary from centralizing the nodes controlled byhimself into a certain shard. Similarly, robustness is to ensure the final execution of the assignment operation.Generally speaking, the epoch randomness 𝜉 𝑒 is treated as a random seed for a pseudorandomness generator[84], to produce multiple pseudorandomnesses for each new node as a reference to be assigned.16 omponent 3: Na (Node Assignment) input: selected nodes 𝑠𝑛𝑜𝑑𝑒𝑠 , epoch randomness 𝜉 𝑒 . output: assigned nodes 𝑎𝑛𝑜𝑑𝑒𝑠 = { 𝑎 , · · · , 𝑎 𝑚 } where 𝑎 𝑗 = { 𝑃 𝑖 } | 𝑘 | for every 𝑗 = 𝑚 . function: assign selected nodes into 𝑚 different subsets randomly based on the epoch randomness 𝜉 𝑒 . property: random distribution; robustness.Note that the list 𝑎𝑛𝑜𝑑𝑒 is not the same as the list of nodes participating in the entire protocol in thenext epoch, since different sharding blockchains might have different replacement rules to substitute the oldnodes in each shard with new nodes. The replacement rule is determined by the Sr component and will beintroduced in Section 4. Component 4 shows the interfaces, functions and properties of the intra-shard consensus
Isc . Isc takescontinuous proposals as inputs, and each proposal could be represented by 𝑝 with a different subscript.In normal blockchains, Isc is used to process transactions, which means the proposals are transactions.In sharding blockchains, a proposal 𝑝 could be a transaction, a transaction input, or other values to becommitted. Isc outputs (cid:104) 𝑝 (cid:105) , where the notation (cid:104)·(cid:105) denotes that the value is committed. Component 4:
Isc (Intra-Shard Consensus) input: a proposal 𝑝 . output: a committed (cid:104) 𝑝 (cid:105) . function: shard members run some consensus algorithm to commit proposals, i.e., reach agreementon proposals. property: consistency; liveness.The basic function of Isc is to process and commit proposals.
Isc might be divided into strong consistency(ref. Definition 20) and weak consistency (ref. Definition 16), depending on the specific algorithm adopted.Strong consistency corresponds to classic distributed consensus algorithms, such as BFT-type algorithms,while weak consistency corresponds to PoW, PoS, and other methods.Regardless of the specific implementation method adopted by
Isc , Isc should meet the consistency andliveness properties. Consistency ensures that all honest nodes have an identical view of the commitment val-ues and liveness guarantees that
Isc could process any proposal within a period of time. In the asynchronousnetwork transmission model, since there is no GST, when the network condition is bad, the protocol canonly choose either consistency or liveness. As an intra-shard consensus algorithm for sharding blockchains,the assurance of consistency is more important.
As shown in Component 5,
Cstp takes in a transaction package
𝑇 𝑋 𝑠 as input. Note that in practicalsituations, transactions may be uploaded individually by users, and different transactions are submitted tothe corresponding shard. The output of
Cstp is a transaction log denoted by
LOG 𝑐 which is specific to thecurrent shard. Cstp includes and makes use of
Isc . The basic function of
Cstp is to process cross-shard transactions.Note that cross-shard transactions occupy most of the transactions in a sharding blockchain, and intra-shard transactions could also be processed by
Cstp as special cross-shard transactions. In most shardingblockchains, the processing of cross-shard transactions could be divided into two phases. In the first phase,input shards generate proofs to prove if the transaction inputs are available or not and send the proofs torelated shards. In the second phase, all related shards verify if the transaction is valid through the proofsreceived. Please see Section 8 for a detailed analysis.17 omponent 5:
Cstp (Cross-Shard Transaction Processing) input: a transaction package 𝑇 𝑋 𝑠 . output: 𝑚 committed transaction logs LOG , · · · , LOG 𝑚 . function: Cstp invokes
Isc inside each shard to process and commit cross-shard transactionsthrough interactions and communication with other related shards. In each shard,
Cstp outputsthe corresponding transaction log or blocks. property: common prefix inside a shard; no conflict between shards; liveness.The properties that Cstp needs to satisfy are the same as those of a secure sharding blockchain definedin Definition 25, that is, consistency and liveness. Due to the special scenario of the sharding blockchain,the consistency property includes common prefix inside a shard and no conflict between shards.
As shown in Component 6, Sr takes in the assigned nodes 𝑎𝑛𝑜𝑑𝑒𝑠 and the list of epoch 𝑒 list 𝑒 as inputs,and outputs the shard member list list 𝑒 + of epoch 𝑒 + C = { 𝑃 𝑖 } | 𝑢 | is a set including 𝑢 nodes. Note thatwe use C to denote committee members when there is a committee in a shard, while C could also be usedto denote shard members when there is no committee in a shard. Component 6: Sr (Shard Reconfiguration) input: assigned nodes 𝑎𝑛𝑜𝑑𝑒𝑠 , a shard member list list 𝑒 of epoch 𝑒 . output: a shard member list of epoch 𝑒 + list 𝑒 + = { C 𝑒 + , · · · , C 𝑚𝑒 + } where C 𝑗𝑒 + = { 𝑃 𝑖 } | 𝑢 | for every 𝑗 = 𝑚 . function: confirm the shard member list of epoch 𝑒 + list 𝑒 and 𝑎𝑛𝑜𝑑𝑒𝑠 , i.e., determinewhich old nodes of each shard are replaced by new nodes; specify the details of bootstrapping whennew nodes join the shard. property: honest shard; liveness.Due to the adversary corruption attack, shards or committees need to be updated after a certain periodof time, or an adversary might control a shard. The basic functions of Sr are to determine which nodesparticipate in the protocol in epoch 𝑒 +
1, namely the members of each shard. Generally, in order toensure that the blockchain can still process transactions normally during reconfiguration, only part of theold members are replaced with new ones during reconfiguration. The replacement process may be random(epoch randomness 𝜉 𝑒 is useful in this case, as shown in Fig. 4), or rely on other special rules. Please referto Section 2.1.2 for details. In addition, Sr needs to design the bootstrapping details when a new node joinsthe shard, such as downloading historical transaction data and UTXO/account data.One of the properties that Sr needs to satisfy is to ensure that each shard is honest. An honest shardmeans that the honest member proportion in each shard exceeds the preset safety threshold, which isdetermined by the Isc component inside the shard. For instance, 𝑢 ≥ 𝑓 + Isc adoptsPBFT [43] as its basic algorithm.
As shown in Component 7, Mm does not have clear inputs and outputs, since Mm is usually determinedby the entire blockchain protocol, rather than as a local algorithm that could be called by nodes.Generally, a motivation mechanism includes an incentive mechanism to reward the active and honestnodes, as well as a punishment mechanism to collect fines from nodes who behave maliciously or go offline.The penalty mechanism might first require each participating node to pay a certain deposit, and all nodescould report the malicious behaviors of other nodes.18 omponent 7: Mm (Motivation Mechanism) function: reward nodes who participate the protocol positively and honestly; punish nodes whobehave negatively and maliciously. property: fairness. Mm needs to meet the fairness of reward distribution, i.e., assuming that the nodes participating inthe protocol are rational, the level of rewards for nodes should correspond to their workload. For example,committee leaders usually have higher computation and communication costs and deserve a higher reward. The previous section gives the components of sharding blockchains, including their interfaces, basic func-tions, and properties. Now, we could utilize these components to compose a complete sharding blockchainsystem.
A complete sharding blockchain system includes all the building blocks described earlier in the previoussection. In the following, we discuss how to compose all these building blocks into a complete shardingblockchain system.
Figure 5: Component composition diagram of a sharding blockchain.
For
Cstp and
Isc , it is clear that the actors are committees. For other components, i.e., Ns , Er , Na , and Sr , different blockchainprotocols might have different actors. For example, in RapidChain, there is a reference committee as an actor. In Omniledger, it is allthe participants in the protocol as actors. So we do not mark the actors of these components in the figure. A complete sharding blockchain protocol Π is a composition of the Ns , Na , Er , Sr , Isc , Cstp , and Mm components, and the composition approach is shown in Fig. 5. Besides, Fig. 5 describes the epoch transitionoperations of a sharding blockchain, i.e., from epoch 𝑒 to epoch 𝑒 +
1. Since Mm is employed by the entireprotocol rather than a local algorithm of nodes, Mm is not indicated in Fig. 5. We divided the wholeoperations of a sharding blockchain into the following two parts.The first part is the confirmation of the shard member list for epoch 𝑒 +
1, including the Ns , Na , Er and Sr components. First, Ns adopts a certain method, such as PoW, PoS and CA, to selects a certain numberof qualified nodes 𝑠𝑛𝑜𝑑𝑒𝑠 among all the participating nodes 𝑝𝑛𝑜𝑑𝑒𝑠 . Second, at the end of epoch 𝑒 , Er is19 axonomy of Sharding Blockchains NodeSelection EpochRandomness NodeAssignment Intra-ShardConsensus Cross-ShardTransaction Processing ShardReconfiguration MotivationMechanism
PoW-based PoS-based [88]
CA-based [86]
Underlyingblockchain [58]
Referencecommittee [31, 59]
VRF [89]
Thresholdsignature [90]
PVSS [58, 59]
Hash [31, 91]
VDF [92]
Others [93, 94]
Binomial [58]
Hypergeometric [59, 95]
Others [96]
Strongconsistency WeakconsistencyPartialsync.Sync. [97, 98]
Async. [99]
Paxos+ [100]
PBFT+ [43, 101]
PoW [60]
PoS [89] [59]
Relaytransaction [60]
Client-driven [58]
Shard-driven [102]
Randomreplacement [58]
SpecificrulesChronological [56, 103]
Boundedcuckoo [104, 105]
Penalty [52]7
Rewards [106]
Reputation [59]
Instantsharding EventualshardingPermissionlesssharding Permissionedsharding
Figure 6: Taxonomy of each component. run to generate a secure epoch randomness 𝜉 𝑒 . Third, Na uses 𝜉 𝑒 to randomly allocate all new nodes of 𝑠𝑛𝑜𝑑𝑒 into 𝑚 different groups, corresponding to 𝑚 shards. Note that at this time, 𝑎𝑛𝑜𝑑𝑒𝑠 only includes newnodes, which means 𝑎𝑛𝑜𝑑𝑒𝑠 is not equivalent to the list of shard members for the new epoch. At last, Sr takes in 𝑙𝑖𝑠𝑡 𝑒 and 𝑎𝑛𝑜𝑑𝑒𝑠 as inputs, determines which old members are to be replaced by new nodes in eachshard, and finally generates a shard member list for epoch 𝑒 +
1. Note that Ns might take place during theexecution of the entire epoch 𝑒 , while Na , Er , and Sr are usually executed during epoch changes. Besides,the above operations apply for every epoch.The second part is related to transaction processing, including Isc and
Cstp . Each shard runs the
Isc component. If the
Isc has the strong consistency property, then there is a committee inside each shardto run
Isc . Cstp processes transactions by invoking
Isc . Note that the inputs from
Cstp to Isc are notalways transactions, but also other kinds of proposals, e.g., transaction inputs. In addition, inter-shardcommunication is required among shards to complete the commitments of cross-shard transactions. Finally,each shard outputs its own transaction log, i.e.,
LOG , · · · , LOG 𝑚 . In the process of constructing a sharding blockchain system, by choosing distinct kinds of system modelsand components, we could obtain different types of sharding blockchain systems.
Distinct system models.
As shown in Fig. 3, system models include network models, adversary models, andtransaction models.For the message transmission model, a sharding blockchain usually adopts a partially synchronous or asynchronous model. Note that the message transmission model for the entire network and inside each shardin a sharding blockchain could be different.Regarding the node admission model, by choosing different models, we could obtain permissionless andpermissioned sharding blockchains. In a permissioned network, nodes that participate in the protocol firstneed to register their identity in a CA. The process of identity registration by the CA could be regarded as aparticular way for the permissioned sharding blockchains, e.g., [85, 86, 87], to implement the Ns component.For the adversary corruption model, a sharding blockchain usually assumes an adaptive and mild ad-versary model, since an immediate corruption adversary is much too powerful and unrealistic. For theproportion model, the total and intra-shard proportion models are related. The intra-shard proportionmodel should first be determined according to Isc . Then, the total proportion should be lower than theintra-shard proportion, since in the process of Na and Ns , an adversary might increase its proportionthrough various attacks. 20or the transaction model, the UTXO model and the account model could be commonly used. TheUTXO model supports multiple input and multiple output transactions. The account model usually onlysupports single-input single-output transactions. Distinct components.
For each component, we could choose specific and different implementation methods.In Section 4-10, we will classify the possible implementation methods of each building block and introducethe basic concepts, existing approaches, and possible problems. Here, we give the taxonomy of each buildingblock in Fig. 6.In Fig. 6, it is worth noting that in the
Isc part, according to the specific algorithm used, the shardingblockchains could be divided into instant sharding blockchains and eventual sharding blockchains.
Definition 28 (An Instant Sharding Blockchain).
In a sharding blockchain, if there is a committeerunning a consensus algorithm with strong consistency inside each shard to process transactions, then it iscalled an instant sharding blockchain.
In instant sharding blockchains, the transaction confirmation is instant, due to the strong consistencyproperty of the intra-shard consensus algorithm. ELASTICO [31], Chainspace [102], Omniledger [58], Rapid-Chain [59], RSCoin [107], etc. are all instant sharding blockchains.
Definition 29 (An Eventual Sharding Blockchain).
In a sharding blockchain, if there is no committeeinside a shard, and the intra-shard consensus algorithm satisfies weak consistency, then it is called an eventualsharding blockchain.
Eventual sharding blockchains are relative to instant ones, where each shard still relies on PoW, PoS, orother methods to confirm transactions. Transactions or blocks are not confirmed instantly, and several blocksat the end of a blockchain must be removed to obtain stable states. The number of blocks is determined bythe system security parameter. The cross-shard transaction processing in eventual sharding blockchains isdifferent from the one in instant sharding blockchains due to its weak consistency property in each shard.Eventual sharding blockchains include Monoxide [60], Parallel Chains [89], etc.
Next, we take Omniledger [58] as an example to illustrate how to use our proposed functional componentsto compose a complete sharding blockchain system.The system models should first be analyzed. Since Omniledger uses a partially synchronous BFT algo-rithm within the shard, the entire message transmission model is a partially synchronous network. Besides,Omniledger allows nodes to join the protocol dynamically, so the node admission model is a permissionlessnetwork. Regarding the adversary model, Omniledger assumes the adaptive and mild adversary adoptedby most blockchains. The intra-shard proportion model is 𝑢 = 𝑓 +
1, which is determined by
Isc . As aresult, the total proportion is limited to [ , / ) . Note that since Omniledger utilizes an underlying PoWblockchain to realize Ns , the honest chain quality decreases due to the selfish mining attack. Consequently,the actual total computational power proportion of the adversary is constrained to [ , / ] . This will beanalyzed in detail in Section 4.2.1.Next is the implementation method for each component. The first is the Ns component. In epoch 𝑒 ,Omniledger requires all nodes that want to participate in epoch 𝑒 + 𝑠𝑛𝑜𝑑𝑒𝑠 . Second,Omniledger uses Randhound (described in detail in Section 5) as the randomness generation component Er .A leader of Randhound is elected by Verifiable Random Functions (VRF), and then all participating nodesexecute PVSS for multiple rounds of interactive communication to generate a secure epoch randomness Note that Casper FFG is not a sharding blockchain. We refer to it here since as far as we know, there is currently nosharding blockchain with a penalty mechanism, and the penalty mechanism of Casper FFG could be seen as a reference. 𝑒 . Third, the Na component takes in 𝐻 ( || 𝜉 𝑒 ) as a seed to compute a pseudorandom permutation 𝜋 ,𝑒 ,and assigns the selected nodes into 𝑚 different groups to obtain 𝑎𝑛𝑜𝑑𝑒𝑠 based on 𝜋 ,𝑒 . Fourth, the shardreconfiguration component Sr stipulates that log 𝑛 / 𝑚 old members in each shard are replaced. Similarly, Sr uses 𝐻 ( 𝑐 || 𝜉 𝑒 ) as a seed to generate 𝑚 pseudorandom permutations 𝜋 ,𝑒 , · · · , 𝜋 𝑚,𝑒 for each shard, and thendetermines which old members are randomly replaced by the assigned new nodes in 𝑎𝑛𝑜𝑑𝑒𝑠 . The followingparts are about transaction processing. Omniledger employs an improved version of PBFT which is calledOmnicon, as the implementation of Isc . Regarding
Cstp , Omniledger utilizes a client-driven 2PC methodto process cross-shard transactions, where the client acts as a coordinator to complete the collection andforwarding of proofs for transaction inputs [58].In this way, we obtain a complete sharding blockchain protocol. We argue that our components andtheir outlined composition are suitable for most sharding blockchain systems. The independent design andcomposability of each component allow our framework to be used to conceptualize and develop new, yetunexplored sharding blockchain systems.
We provide a summary of sharding blockchain systems in Table 2.The notation “ (cid:51) ” means that the system has the property; “ (cid:55) ” means the system does not have theproperty; “-” denotes that the property does not apply to the system. We use “SGX-Sharding” to denotethe system proposed in [95] which utilizes the trusted hardware Intel SGX. Similarly, PoSBP represents thesystem described in [88].The network model for RapidChain is partially synchronous for the whole network and synchronous insidea committee. In the “scalability” line, ELASTICO is not scalable since all transactions will be processed bya final committee. Monoxide is not scalable since all miners have to verify all transactions in the network,which is analyzed in Reference [108].
4. Node Selection
In this section, we first introduce basic concepts to realize node selection for sharding blockchains inSection 4.1. Then existing approaches to select new nodes are classified into PoW-based ones and PoS-based ones in Section 4.2. In addition, PoW-based methods consist of using an underlying blockchain andusing a reference committee. Finally, we analyze potential problems in the process of node selection inSection 4.3.
Node selection is necessary for sharding blockchains to select qualified shard members. The problems tobe solved in the node selection process are as follows. First, all nodes should have a consistent view of theselected result, i.e., 𝑠𝑛𝑜𝑑𝑒 . Second, the specific requirements that honest node number proportion in 𝑠𝑛𝑜𝑑𝑒 should meet for different application scenarios need to be analyzed. Third, to against various attacks, strictsecurity proofs should be given for the node selection process..In permissioned networks, the node selection process is completed with the participation of CA throughproviding the identity registration service for nodes. In permissionless networks, the node selection processis more complicated, so we discuss this situation in detail.During the selection process in a permissionless network, PoW or PoS is utilized to prevent Sybil attacks[83, 109], that is, an adversary increases the probability of becoming a shard member by creating fakeidentities. If a PoW-based node selection method is adopted, a certain mining difficulty needs to be setcarefully, such that enough nodes could find PoW solutions in each period to replace the corresponding oldones. In a PoS-based node selection method, a certain number of nodes need to be selected randomly asnew shard members according to the stake held by each node.The node selection process usually causes a decline in the proportion of honest nodes for some reason. Inorder to measure the degree of decline, we introduce an honest fraction decline degree parameter 𝜔 𝑑 , whichis described in Definition 30. 22 able 2: Summary of sharding blockchain systems. System ELAS-TICO [31]
Omniledger [58]
Rapid-Chain [59]
Chainspace [102]
SGX-Sharding ♯ [95] ZILLIQA [91]
PoSBP ♯ [88] Ethereum [92]
Monoxide [60]
ParallelChains [89]
RSCoin [107] C l a ss i fi c a t i o n NodeAdmissionModel
Permission-less Permissionless Permissionless Permissionless Permissioned Permissionless Permission-less Permissionless Permissionless Permission-less Permissioned
Instant orEventual
Instant Instant Instant Instant Instant Instant Instant Instant Eventual Eventual Instant S y s t e m M o d e l NetworkModel
PartiallySync. PartiallySync. PartiallySync./Sync. * PartiallySync. PartiallySync. PartiallySync. PartiallySync. PartiallySync. PartiallySync. PartiallySync. -
AdversaryModel ≤ ≤ ≤ - ≤ ≤ ≤ ≤ ≤ ≤ - TransactionModel
UTXO UTXO UTXO Account Generic ♮ Account UTXO Account Account UTXO Account N o d e S e l e c t i o n a nd A ss i g n m e n t SybilattacksResistance
PoW PoW PoW - SGX PoW PoS PoS PoW PoS -
BasicMethod
ReferenceCommittee UnderlyingBlockchain ReferenceCommittee - SGX ReferenceCommittee Hash Func. VDF - VRF -
DistributionModel - Binomial Hypergeomet-ric - Hypergeomet-ric - - - - - -
Epoch Randomness
Hash Func. RandHound(PVSS) VSS - SGX Hash Func. - RAN-DAO+VDF - VRF - I n t r a - Sh a r d C o n s e n s u s AdversaryModel 𝑢 = 𝑓 + 𝑢 = 𝑓 + 𝑢 = 𝑓 + 𝑢 = 𝑓 + 𝑢 = 𝑓 + 𝑢 = 𝑓 + 𝑢 = 𝑓 + 𝑢 = 𝑓 + ≤ ≤ 𝑢 = 𝑓 + ConsensusAlgorithm
PBFT Omnicon(BFT) Sync BFT PBFT AHL (BFT) PBFT BFT-DPoS BFT PoW PoS BFT C r o ss - Sh a r d S c h e m e BasicAlgorithm - 2PC Split 2PC 2PC - - RelayTransaction RelayTransaction - 2PC
Coordinator - Client-Driven Shard-Driven - Shard-Driven - - - - - Client-Driven Sh a r d R e c o nfi g u r a t i o n Basic Rule - RandomReplacement BoundedCuckoo Rule - RandomReplacement - - - - - -
UpdateFraction - log 𝑢 - log 𝑢 - - - - - - P e r f o r m a n c e Responsive-ness (cid:55) (cid:55) (cid:51) (cid:51) (cid:55) (cid:55) (cid:55) (cid:55) (cid:55) (cid:55) (cid:55)
Scalability (cid:55) (cid:5) (cid:51) (cid:51) (cid:51) (cid:51) (cid:55) (cid:55) (cid:51) (cid:55) (cid:5) (cid:55) (cid:55)
The notation “ (cid:51) ” means that the system has the property; “ (cid:55) ” means the system does not have the property; “-” denotes that the property does not apply to the system. ♯ We use “SGX-Sharding” to represent the system proposed in [95] which uses the trusted hardware Intel SGX. Similarly, PoSBP represents the system described in [88]. * The network model for RapidChain is partially synchronous for the whole network and synchronous inside a committee. (cid:5)
ELASTICO is not scalable since all transactions will be processed by a final committee. Monoxide is not scalable since all miners have to verify all transactions in the network, which is analyzed inreference [108]. ♣ “Malicious leader resistance” refers to the ability to prevent a malicious adversary from providing false input availability certificates (ref. Definition 32) during cross-shard transaction processing. ♮ As explained in Section 2, “Generic” means a UTXO model or an account model. efinition 30 (Honest Fraction Decline Degree). Assume that the honest fraction (computational poweror stake) is 𝛽 . After a node selection process, let newnodes denote the selected node list. Assume the honestnode fraction in newnodes to be ( − 𝜔 𝑑 ) 𝛽 , then 𝜔 𝑑 is said to be the honest fraction decline degree parameter. In order to ensure that the proportion of honest nodes in the node selection process will not decreasetoo much, we describe the concept of fair selection, which is defined in Definition 31.
Definition 31 (Fair Selection [110]).
Let 𝑄 𝑓 denote the fraction of honest nodes in a selected node list newnodes and let 𝛽 denote the honest fraction (computational power or stake). We say that a node selectionprocess for shard members is ( 𝑘 𝑓 , 𝜔 𝑑 ) -fair if for all 𝛽 > , there exists some negligible function 𝜇 𝑓 ( 𝑘 ) suchthat for every 𝑘 ≥ 𝑘 𝑓 , ≤ 𝜔 𝑑 < , the following condition holds Pr [ 𝑄 𝑓 ≥ ( − 𝜔 𝑑 ) 𝛽 ] ≥ − 𝜇 𝑓 ( 𝑘 ) The definition of “fair selection” in [110] is inspired by [49], while it has some obvious differences fromthe definition of “fairness” in FruitChains. Fairness in FruitChains only applies to its own specially designedmining process, where a fruit and a block are mined simultaneously through a single 2-in-1 mining function.So the analysis in FruitChains is specific. Definition 31 gives a more general description of the node selectionprocess, which could be used to evaluate the fairness of a selection result.
According to the underlying technologies of node selection methods, we divide related approaches intothree categories, i.e., PoW-based, PoS-based, and CA-based node selection methods. As mentioned inSection 3.2, the first two methods are suitable for permissionless networks, and the latter method is used inpermissioned networks.
A PoW-based node selection approach utilizes PoW mining to select qualified nodes where any nodewho wants to take part in the protocol must find a PoW solution. There are currently two PoW-based nodeselection methods, namely, using an underlying blockchain and using a reference committee.
Using an underlying blockchain.
As shown in Fig. 7, an underlying blockchain is a chain similar to that ofBitcoin [1]. All blocks are connected by hash pointers. Nodes mine on the basis of the last block, and verifyif the hash value meets the requirements according to the following condition: ℎ = H ( 𝑠𝑡𝑟, 𝑛𝑜𝑛𝑐𝑒, 𝑝𝑘 𝑖 ) < 𝐷 Figure 7: PoW-based node selection: using an underlying blockchain.
Here, 𝑠𝑡𝑟 is the hash of the last block, and 𝑛𝑜𝑛𝑐𝑒 ∈ { , } 𝜆 denotes the potential solution of the PoW. 𝜆 is a security parameter where 𝜆 ∈ N . 𝑝𝑘 𝑖 is the public key of the node 𝑃 𝑖 . 𝐷 is a difficulty parameter where24 = 𝑝 · 𝜆 and for all ( 𝑠𝑡𝑟, 𝑛𝑜𝑛𝑐𝑒, 𝑝𝑘 𝑖 ) , we have Pr [ H ( 𝑠𝑡𝑟, 𝑛𝑜𝑛𝑐𝑒, 𝑝𝑘 𝑖 ) < 𝐷 ] = 𝑝 . 𝑝 is the probability that onenode finds a PoW solution successfully in one single round.After finding a nonce that meets the requirement, a node broadcasts the block, i.e., ( 𝑠𝑡𝑟, 𝑛𝑜𝑛𝑐𝑒, 𝑝𝑘 𝑖 ) .Then the nodes receiving the block verify its legitimacy. If the requirements are met, then the nodes willcontinue to mine, using H ( 𝑠𝑡𝑟, 𝑛𝑜𝑛𝑐𝑒, 𝑝𝑘 𝑖 ) as a new 𝑠𝑡𝑟 . The block producers, that is, nodes that find validPoW solutions successfully, are considered as new shard members. When there are enough shard membersconfirmed, shards could launch a reconfiguration to update members.Note that the node selection process is necessary for all hybrid consensus blockchains, which combinesclassical distributed consensus algorithms and the blockchain consensus, such as PeerCensus [55], ByzCoin[56], Solida [57], Hybrid Consensus [76], Thunderella [111], and Algorand [72]. In Solida [57] and Byzcoin[56], a committee conducts a reconfiguration whenever a new block producer is confirmed through the abovesteps.Omniledger [58] uses an underlying blockchain to select new members. Specifically, in epoch 𝑒 , the nodethat wants to participate in epoch 𝑒 + Using a reference committee.
Another approach uses a reference committee and a fixed PoW puzzle to selectnew nodes, which is shown in Fig. 8.
Figure 8: PoW-based node selection: using a reference committee.
This approach includes two steps. First, a mining step. In a current epoch, nodes use a puzzle speciallyset to mine. The mining equation is similar to that of using an underlying blockchain. ℎ = H ( 𝑝𝑢𝑧𝑧𝑙𝑒, 𝑛𝑜𝑛𝑐𝑒, 𝑝𝑘 𝑖 ) < 𝐷 A node submits his solution to a reference committee C 𝑅 after finding a PoW solution successfully, yetthe found PoW solutions do not form a chain. That is, in an entire epoch, the 𝑝𝑢𝑧𝑧𝑙𝑒 remains unchangeduntil a sufficient number of PoW solutions are found. Note that in this case, a miner might use a differentpublic key to continue mine after finding a PoW solution. This will not influence the system security aslong as the adversary’s computational power is limited.Second, a new node list confirmation step. After enough number of PoW solutions are submitted, thereference committee runs an intra-committee consensus to confirm the new node list, denoted by newnodes .The intra-committee consensus might be a BFT style consensus algorithm, where a leader is responsible forproposing a value, and other nodes vote for the proposal. After an agreement is reached upon the new nodelist, the reference committee broadcasts the committed newnodes to the entire network. Nodes that are onthe committed list are regarded as valid members of the next epoch.RapidChain utilizes a reference committee to select new nodes [59]. The puzzle in RapidChain is a freshrandomness generated by the reference committee based on verifiable secret share (VSS). An adversary couldnot precompute a PoW solution ahead of honest nodes since the randomness is unpredictable. Besides, thePoW mining process is done offline without influencing the normal operations of the whole protocol. At the25eginning of every epoch, the reference committee reaches an agreement on a reference block which includesthe new node lists for the epoch. PoS-based node selection approaches have the following characteristic: the more stakes a user has, thehigher the probability of being selected. In general, the coins in the system need to be divided into smallunits, and each unit has the right to participate in the selection of committee members. In this way, it isensured that the probability of being selected as a committee member will increase if a node has more stakes.At the same time, when judging whether each unit is selected, the verifiable random function (VRF) couldbe employed. The unique serial number of each unit could be regarded as the input of VRF to generate therandom output and its corresponding proof. Whether a unit is selected as a committee member could beverified publicly by judging if the random output satisfies a certain threshold condition.Similar to PoW-based node selection approaches, there are two fundamental ways to use PoS to selectcommittee members, i.e., using an underlying blockchain or not.
With an underlying blockchain.
Using an underlying blockchain means that nodes still rely on PoS to gen-erate blocks first, and then block producers during a certain time range are considered to be committeemembers. Some PoS-based underlying blockchains, such as Ouroboros [54], can be used to confirm com-mittees in a sharding blockchain. The basic process is that the block producers within a period of time areconfirmed as new nodes, and the new nodes are randomly allocated to multiple different shards based on arandomness.
Without an underlying blockchain.
If there is no underlying blockchain, using PoS to select shard membersrequires selecting multiple nodes at once, such as a committee or multiple committees.Lee et al. [88] propose a sharding blockchain system that uses PoS to select committee members. Theselection process is quite simple, which uses the following formula H ( 𝑣 ’s address || H ( 𝑏 )) mod 𝑘 . H is a hashfunction, 𝑣 represents a validator, 𝑏 denotes the last block in the previous epoch, and 𝑘 is the number ofshards. In this way, validators in [88], i.e., nodes, are assigned into 𝑘 shards randomly. Ethereum sharding[92] adopts the PoS-based node selection method to select shard members.Parallel Chains [89] is an eventual sharding blockchain system that uses PoS to directly generate blocks,while there is no committee in each shard. The blockchain in each shard is built on top of that of OuroborosPraos [112]. In the permissioned network, the function of the Ns component is realized by a CA. Every node thatwants to participate should first complete identity authentication via the CA. When there are enough nodes(in the initial phase of the protocol), or at the end of each epoch (during epoch reconfiguration), the CA willpublish a list of nodes participating in the protocol, i.e., 𝑠𝑛𝑜𝑑𝑒 , which contains the public key information ofnodes. Only the nodes on the list published by the CA can take part in further operations of the protocol, i.e.,join different shards and process transactions. The permissioned sharding blockchains that use CA-basednode selection approaches include [85, 86, 87]. We summarize problems that might occur during the process of node selection adopting both PoW-basedand PoS-based node selection approaches in the following.
We analyze the potential problems in each kind of PoW-based node selection approach. First, possibleproblems of using an underlying blockchain approach are introduced, including the impact of attacks (e.g.,selfish mining, stubborn mining, etc.). Then potential threats of using a reference committee are analyzed,including an adversary’s potential mining advantages and node censorship attacks by malicious leaders.26 sing an underlying blockchain.
When an underlying blockchain is used to select nodes, an adversary couldlaunch the attacks such as selfish mining [113, 114], stubborn mining [115, 116], block withholding [117], andfork after withholding [118], to increase his proportion of blocks and get more advantages to be selected. Inthis case, the chain quality of honest nodes decreases severely.The selfish mining attack is shown in Figure 9. In a selfish mining attack, an adversary and honest nodesmine at the same time. After finding a PoW solution (i.e., a block), an honest node broadcasts the block tothe entire network immediately. On the contrary, an adversary does not broadcast a block after finding aPoW solution, yet adopts different strategies to reveal his blocks in different situations. The blockchain thatis known to all honest nodes is called the public chain, while we name an adversary’s privately controlledblockchain as a private chain. When the adversary’s private chain is longer than the public chain, theadversary does not immediately announce the block after finding a new PoW solution, i.e., withhold theblock. When the length of the public chain catches up with that of the private chain, the adversary publishesa certain number of private blocks, making the new chain longer than the public chain. In this way, honestnodes will choose to continue mining at the end of the newly disclosed chain. Selfish mining increases theproportion of blocks controlled by an adversary. Intuitively, an adversary could waste honest computationalpower. When the adversary’s private chain is longer, he does not publish the block. At this time, even if avalid PoW solution is found by honest nodes, it will be replaced by an adversary’s longer chain.
Figure 9: Schematic diagram of selfish mining.
The stubborn mining attack [115] provides an attacker with more advantages in PoW mining. Stubbornmining is actually an extension of selfish mining, which wastes honest computational power by creatingmore opportunities for competition. Three basic strategies and their different combinations are proposedand analyzed in depth [115]. In addition, the block withholding attack [117] and fork after withholdingattack [118] are proposed. We regard the attacks above as the same type of attack as selfish mining. Byformulating different mining strategies, an adversary could occupy a higher proportion of all blocks generatedthan it deserves. In other words, the chain quality of honest nodes decreases.Besides, the eclipse attack [119, 120] and network partition attack [121] could also be used to enhancethe effects of selfish mining type attacks [115]. The key idea of the eclipse attack is to control all incomingand outgoing connections of a node [119]. The network is divided into three partitions, i.e., an attacker, avictim, and an honest one’s part. Although some measures could detect intrusion [122], eclipse attacks arestill appealing for attackers.These kinds of attacks decreases the honest chain quality while increasing the fraction of blocks belongingto an adversary. A formal analysis is given in the notable Bitcoin backbone protocol [18] which points outthat the ratio of blocks contributed by an adversary is bounded by 𝑡𝑛 − 𝑡 where 𝑡 and 𝑛 is the computational power of an adversary and the entire network, respectively. For example,when 𝑡 = / 𝑡 /( 𝑛 − 𝑡 ) equals to 1 /
2. When 𝑡 = / 𝑡 /( 𝑛 − 𝑡 ) = /
3. Hence, a 3 / / Using a reference committee.
In a PoW-based member selection process, two important factors influencethe selection results, i.e., the time advantages of an adversary to mine, and the confirmation of a new node27ist. We give a detailed analysis in the following.The time advantages of an adversary to mine mainly refer to the network latency. The adversary isusually responsible for network transmission who might have a certain advantage in mining. In a partiallysynchronous network, the adversary could delay messages sent by honest nodes for at most Δ time, whichis the upper bound of the network delay. On the one hand, the adversary could acquire a mining puzzlein advance so that he could start mining ahead of the honest nodes. On the other hand, an adversarycould delay honest miners’ PoW solutions submitted to a reference committee. The adversary’s networkadvantage is a key factor that must be considered when strictly analyzing the mining process and results. Inthe analysis of the mining step, the speed to find a PoW solution needs to be calculated. The mining stepcould be treated as independent binary random variables. The value for each variable is 1 with probability 𝑝 . So the PoW solutions found by honest nodes or the entire network could be estimated by the Chernoffbound [123].In any case, the adversary could have a certain time advantage over honest nodes to obtain a miningpuzzle. Since even if an unpredictable randomness (see Section 5.1) is used, an adversary has the advantageof network latency. If there is no randomness used as a puzzle, the adversary might learn a puzzle in advancethrough various other means such as withholding his PoW solutions. An adversary’s time advantage mustbe considered in the analysis of the mining process. Because the adversary will produce more PoW solutionsthan expected. To ensure that the proportion of honest nodes in the finally found PoW solutions exceedsa certain safety threshold, it is usually required that the total number of solutions exceeds a certain lowerbound. A concrete analysis of the mining step is referred to [110].The other factor to consider is the confirmation of a new node list. After enough number of PoWsolutions are found, a reference committee run an intra-shard consensus to confirm newnodes . The mostcommonly employed intra-shard consensus is the BFT-type algorithm. In such a BFT algorithm, there is aleader who proposes a new node list. A malicious leader could “censor” and exclude some honest nodes, toinvolve more malicious nodes, and thus harm the system security.The specific procedures of the node censorship attack are as follows. Let newnodes and newnodes (cid:48) denotetwo member lists proposed by a leader and held by an honest node in a reference committee C 𝑅 , respectively.A malicious leader might replace a certain number of honest nodes in newnodes with the same number ofmalicious nodes who find the PoW solution after the replaced honest nodes [110].In this case, if honest members in C 𝑅 vote without checking the validity of newnodes , then safety will beruined since the malicious nodes proportion on newnodes will exceed the specified limit. As a result, A willcontrol the committee in the next epoch. On the contrary, if an honest member only votes when newnodes equals to newnodes (cid:48) held by himself, then the liveness property will be broken with a high probability, sincethe new member list held by different honest members may have some differences due to the network latency.The above node-censorship attack could be handled by a threshold-vote strategy proposed in [110]. Aproper threshold 𝑘 𝑇 is chosen for the differences between newnodes received from the leader and newnodes (cid:48) held by an honest committee member. A BFT algorithm needs to be modified in moderation to be compatiblewith the threshold-vote strategy. The specific threshold-vote rule is as follows. An honest committee membervotes for a list newnodes if and only if the number of different nodes between newnodes and newnodes (cid:48) isless than or equals to 𝑘 𝑇 . The threshold-vote strategy might introduce some new problems. For specificanalysis, please refer to [110].The future research directions for PoW-based node selection approaches are as follows. First, design amore fair node selection approach to make the honest fraction decline degree lower. Second, analyze variousattacks that may be encountered in the process of using PoW such as selfish mining attacks. Third, usea strict analysis process to analyze the security of the entire mining process, and the requirement of eachparameter. Fourth, fully consider the characteristics of the sharding blockchain, and design a method forselecting sharding members more suitable for the sharding blockchain. In a PoS-based node selection process, some vulnerabilities or attacks might happen. We separatelydescribe the possible problems in the two types of PoS-based node selection approaches.28 ith an underlying blockchain.
In the following, we describe the attacks against the PoS-based node selec-tion approaches using an underlying blockchain.Nothing at stake [124] refers to that an attacker tries to mine on different forks of the chain to obtainhigher benefits. In a PoS-based blockchain, to generate a fork is not as costly as that in a PoW-basedblockchain, where a huge amount of computational power might be required. In a PoS-based blockchain, ifthere is no protective mechanism, when the blockchain has a fork, a node will try to mine on both forks toincrease the probability of obtaining profit. Nothing at stake could be prevented by introducing punishmentmechanisms in the PoS consensus, that is, punish the nodes that generate blocks on different forks.A grinding attack [125] means that in some PoS consensus mechanisms, the selection of a block producerin the 𝑟 + 𝑟 , or multiple blocks in previous rounds.Namely, the selection results are not random and might be biased. In some PoS consensus mechanisms,the block producer in round 𝑟 + 𝑟 . If theblock producer in round 𝑟 is controlled by the adversary, to become the block producer in round 𝑟 +
1, theadversary might try to continuously generate different new blocks in round 𝑟 , i.e., “grind” the generatedblock until it is conducive to make the adversary become the next block producer. Grinding attacks couldbe prevented by using an unbiasable randomness such as RandHound [126], to determine a block producer.A long range attack [127] means that when an offline node or a new node joins the network, an adversaryforges a blockchain from the genesis block to the latest block, trying to make the newly joined node acceptthis forged blockchain and mine on it. In a PoW-based blockchain, the longest chain rule or the heaviestchain rule [50] is usually used to determine which blockchain is accepted as the valid main chain by allparticipants. Assuming 𝐴 is the main chain, the adversary wants to create a fake chain 𝐵 to make newlyjoined nodes believe that 𝐵 is the main chain. In a PoW-based blockchain, new nodes can easily judge 𝐴 as the main chain by verifying the difficulty of mining in the two chains, since the mining difficulty of theblocks in 𝐴 must be significantly higher than that of 𝐵 . If the adversary wants to forge a chain with similarmining difficulty, he needs a huge amount of computational power, and the attack cost will greatly exceedthe benefits. However, in a PoS-based blockchain, it is much easier to forge a main chain 𝐴 . The adversarycould bribe the nodes to sell the important private keys used in the past without spending too much to forgea fake chain 𝐵 , convincing the newly joined nodes that chain 𝐵 is the main chain. Long-range attacks couldbe prevented by the checkpoint mechanism [52].Ga?i, Kiayias, and Russell [128] propose a stake bleeding attack against the PoS consensus mechanism.The stake bleeding attack is mainly implemented after a successful long-range attack. For a PoS consensussystem that does not use a checkpoint mechanism, after an attacker launches a long-range attack, the newlyjoined nodes believe that the adversary’s chain 𝐵 is the current main chain, and the transaction generatedby the new nodes will be submitted to chain 𝐵 for processing. The adversary has full control of chain 𝐵 ,and could earn a lot of transaction fees and even launches a double-spending attack [129, 130]. Without an underlying blockchain.
The problems that might be encountered in the PoS-based node selectionapproach without an underlying blockchain mainly include the following two aspects. First, the applicationof some cryptographic techniques, e.g., VRF, might bring more computation and communication overhead.Second, the existence of network delay might affect a node’s view of the selected new nodes.The future research directions for PoS-based node selection are as follows. First, the impacts of variousattacks against PoS on the node selection process and results should be fully considered. Second, thecomputation and communication overhead of some cryptographic tools on practical applications need to beanalyzed. For instance, analyze the time cost required to complete a PoS-based node selection, and evaluatethe impact on system liveness during reconfiguration.
5. Epoch Randomness
In this section, we first introduce basic concepts about epoch randomness in Section 5.1. Then existingapproaches to generate randomness are divided into VRF, PVSS, etc. in Section 5.2. Additionally, we com-pare the state-of-the-art distributed random beacon protocols in Section 5.3. Finally, we analyze potentialproblems and future directions about epoch randomness in Section 5.4.29 .1. Basic Concepts
Epoch randomness is important in sharding blockchains. A randomness could be used as a fresh puzzlefor mining and as a seed to achieve random node allocation. The problems that the Er component needsto solve are as follows. First, it is necessary to determine which nodes are responsible for running Er togenerate the randomness. Second, in each epoch, the time point to invoke Er needs to be confirmed. Third,the running time, system overhead, and failure rate of the epoch randomness generation protocol need to befully considered. Fourth, the properties of the randomness generated by Er need to be analyzed to make itapplicable for the sharding blockchain.Randomness generation could be regarded as an independent research field and has been studied for along time. In 1983, Blum [131] first proposed a coin-tossing protocol that aims at generating random valuesbetween two untrusted parties. In the same year, Rabin [132] formalized the concept of a random beacon,which generates fresh random numbers at regular intervals. For a group of nodes in need of continuousrandom numbers, such protocols can be executed to obtain a reliable source of randomness.When a group of untrusted nodes is involved in a consensus protocol, an important issue is how tofairly generate public randomness without trusted third parties. However, there may be several maliciousnodes trying to bias the outputs or forcing the protocol to restart to their advantage. Therefore, thedistributed randomness protocols need some cryptographic building blocks to ensure fairness and security.Considering distributed approaches, random beacon protocols need to meet with the following properties:public-verifiability, unpredictability, bias-resistance, and availability (liveness), as outlined in [65], [126],[133]. • Public-verifiability: Any third party not directly participating in the protocol should also be able toverify the generated values. As soon as a new random beacon value becomes available, all parties canverify the correctness of the new value using public information only. • Unpredictability: Any node (either honest or malicious) should not be able to predict (precompute)future random beacon values. • Bias-resistance: Neither a single node nor colluding nodes can bias (influence) the output value totheir benefit. • Availability/Liveness: Neither a single node nor colluding nodes can obstruct the progress.In addition to the above four properties, some distributed random beacon protocols also have the propertyof guaranteed output delivery [133], [134], which means that any adversary cannot interfere or prevent honestnodes from obtaining the randomness output.In recent years, there has been a substantial amount of new research related to the generation of dis-tributed randomness in academia and industry, which are introduced below.
Current random beacon protocols employ various cryptographic techniques to generate secure random-ness. These techniques are mainly divided into several categories, namely VRF [135], threshold signature[136], publicly verifiable secret sharing (PVSS) [137],[138] and verifiable delay functions (VDF) [139], etc.In this section, we detail those randomness generation methods.
The concept of VRF comes from a pseudorandom oracle [140], which simulates a random oracle from an 𝑎 -bit string to a 𝑏 -bit string using a seed. Formally, there exists a polynomial-time algorithm 𝐹 (· , ·) suchthat 𝑓 𝑠 = 𝐹 ( 𝑠, ·) : { , } 𝑎 → { , } 𝑏 always holds, where 𝑠 denoted the seed. Intuitively, such a pseudorandomoracle is not verifiable. Therefore, Micali et al. [135] proposed a new type of pseudorandom oracle, namedVRF. That is, given an input 𝑥 , the seed-owner should be able to compute the value 𝑣 = 𝑓 𝑠 ( 𝑥 ) and aproof proving the correctness of 𝑣 in polynomial time. The result 𝑣 = 𝑓 𝑠 ( 𝑥 ) is unique and computationallyindistinguishable from a truly random string 𝑣 (cid:48) of equal length. With application to the blockchain, the30ain idea of the VRF is that all nodes (i) use their private keys as part of the input to generate randomnumbers, (ii) output random numbers along with zero-knowledge proofs, and (iii) verify the correctness ofreceived randomness. Each node combines the output of a VRF with other variables (i.e., round numbers),then signs by its own private key. If the resulted randomness is smaller than a pre-defined threshold, thenode can know privately that it is selected as a leader or a committee member.In general, the purpose of a VRF is to generate verifiable and unpredictable random values locally.Combined with a consensus protocol like PoS, it can dynamically adjust the weight of all nodes. So thisstrategy is scalable and suitable for different applications. That is why there are many well-known publicblockchain projects using the VRF as their randomness source, such as Algorand [72], Ouroboros Praos[112], and DVRFs [141] The idea behind threshold cryptographic schemes is to split secret information (i.e., a secret key) andcomputation (i.e., signature generation or decryption) among multiple parties in order to remove the riskof a single point of failure. The difference between a threshold signature and a general digital signature isthat the former is no longer completed by an individual, but by a threshold set of participants. In a ( 𝑡, 𝑛 ) threshold signature scheme, 𝑛 represents the total number of participants, and 𝑡 is the threshold. When anysubset of 𝑡 (or more) participants jointly sign the same message, they can get a signature representing thewhole group, but any 𝑡 − et al. [143] and Dfinity [90] both employ threshold signatures in their constructions. Secret sharing was first proposed by Shamir [144] in 1979, which enables a dealer to split a secret amonga group of participants, each participant obtains a secret share. Shares can be combined to reconstruct thesecret through polynomial interpolation. Note that the secret sharing scheme has an important precondition:both dealers and participants are honest. If some parties are malicious and send invalid shares, the honestparties may not reconstruct the secret. To deal with a corrupted dealer or invalid shares in the reconstructionphase, verifiable secret sharing (VSS) [145] is proposed. In 1996, Stadler [137] proposed PVSS, where anyone(including participants and third parties) can verify the correctness of shares through public informationonly. During the distribution phase, a dealer computes an encrypted share along with a non-interactivezero-knowledge proof (NIZK) [146] for each participant to ensure the validity of encryption. During thereconstruction phase, the participants recover the original secret by publishing the properly decrypted sharesand the NIZK proof showing its correct decryption. The general idea of PVSS-based schemes is that eachnode (i) privately generates a random secret value, (ii) broadcasts a commitment and shares of this secretto all nodes, and (iii) reveals this secret after verification. If a node fails to do so, other honest nodes canjointly recover the secret from the received shares.The schemes including HydRand [133], Scrape [134], Rand ∗ protocol family [126] and ALBATROSS [147]are all based on PVSS. RandHerd [126] also uses collective signing (CoSi) [148] and ALBATROSS [147] isthe first secure random beacon protocol under the universal composability (UC) framework [149].31 .2.4. Hash Functions Several existing solutions generate hash values as randomness through leveraging resources of existingsystems. For example, PoW [1] relies on block hashes as a source of public randomness, proof-of-delay [150]employs a delay function on top of the PoW block hash, and Caucus [151] is designed in the form of a hashchain, which is implemented within a smart contract on Ethereum.
VDF requires a specified number of sequential steps to compute whether or not it is executed on multipleprocessors, then, produces a unique output that can be efficiently and publicly verified. VDF is useful forconstructing randomness beacons from sources such as PoW-based blockchains, in which powerful minerscould potentially manipulate the beacon result by refusing to post blocks, resulting in producing an unfa-vorable beacon output. Therefore, VDFs with a suitable time delay would be sufficient to prevent attacks,miners will not be able to determine the beacon output from a given block before it becomes stale [139].Lenstra and Wesolowski proposed Unicorn [152] with a sufficiently long delay parameter (longer than thetime period during which values may be submitted), even the last party to publish its random value can-not predict the final beacon outcome [139]. RandRunner [153] implements a trapdoor VDF with stronguniqueness and does not require an agreement protocol for the VDF inputs, which achieves much lowercommunication overhead. Additionally, the Ethereum research team [92] plans to use RANDAO [154] andVDF in the Ethereum beacon chain to randomly select block producers. Chia Network [155] plans to useVDFs to support their proof-of-space and time.
In addition to the above typical types, there exists some other solutions that use various encryp-tion schemes to generate randomness, namely homomorphic encryption (HE) [156] and multi-authorityciphertext-policy attribute-based encryption (MA CP-ABE) [157], etc. The homomorphic property of cryp-tosystems allows nodes to operate the ciphertext directly without decryption. Both HE-Rand [93] andHERB [158] implement the threshold version of ElGamal as homomorphic encryption to generate random-ness. Moreover, Zhang et al. [94] define a threshold MA CP-ABE protocol and use it as a commit-and-revealscheme to construct ABERand as a public distributed randomness beacon. In the following, we provide a specific comparison of the state-of-the-art distributed random beaconprotocols. Our comparison mainly focuses on the cryptographic primitives, network models, randomnessproperties, and complexity evaluation. The results are presented in Table 3 wherein 𝑛 denotes the numberof participants in the network, 𝑐 denotes the size of a subset in the specific protocol. Note that thiscomparison not only considers protocols specifically targeted at implementing random beacons, but alsoincludes approaches that provide a random beacon functionality as a byproduct of their intended application[65],[133]. We divide the network models of the analyzed protocols into three categories, namely synchronous,partially synchronous, and asynchronous models (ref. Definition 1-4).PoW-based blockchains, such as Bitcoin and Ethereum, assume a synchronous network model. Proof-of-delay [150] and Caucus [151] are also synchronous since they are built on top of such PoW-based block-chains. In [126], RandHound and RandHerd are designed within a synchronous setting while RandShareis asynchronous. HE-Rand [93] is also synchronous as the protocol is described in rounds. RandRunneris designed for practical deployment scenarios with bounded network delay, while it still ensures liveness,public-verifiability, and bias-resistance even under periods of full asynchrony [153]. We name the protocol proposed in the paper as “HE-Rand”. able 3: Comparison of distributed random beacon protocols T y p i c a l e x s i s t i n g s o l u t i o n s C r y p t og r a ph i c p r i m i t i v e ( s ) N e t w o r k m o d e l T r u s t e dd e a l e r o r D K G r e q u i r e d L i v e n e ss / f a il u r e p r o b a b ili t y U np r e d i c t a b ili t y B i a s - R e s i s t a n c e C o mm un i c a t i o n c o m p l e x i t y C o m pu t a t i o n a l c o m p l e x i t y V e r i fi c a t i o n c o m p l e x i t y Cachin et al. [143]
Threshold Sig. Async. yes (cid:51) (cid:51) (cid:51) 𝑂 ( 𝑛 ) 𝑂 ( 𝑛 ) 𝑂 ( ) Dfinity [90]
Threshold Sig. + VRF Sync. yes ♯ − (cid:51) (cid:51) 𝑂 ( 𝑐𝑛 ) 𝑂 ( 𝑐 ) 𝑂 ( ) Algorand [72]
VRF Partially Sync. no 10 − (cid:37) (cid:55) 𝑂 ( 𝑐𝑛 ) * 𝑂 ( 𝑐 ) * 𝑂 ( ) * Ouroboros Praos [112]
VRF Partially Sync. no (cid:51) (cid:37) (cid:55) 𝑂 ( 𝑛 ) * 𝑂 ( ) * 𝑂 ( ) * Nguyen-Van et al. [93]
HE + VRF Sync. no (cid:51) (cid:51) (cid:51) 𝑂 ( 𝑛 ) 𝑂 ( ) 𝑂 ( 𝑛 ) RandRunner [153]
VDF Async. no (cid:51) (cid:55) § (cid:51) 𝑂 ( 𝑛 ) ♣ 𝑂 ( 𝑇 ) ‡ 𝑂 ( log 𝑇 ) ‡ Ouroboros [54]
PVSS Sync. no (cid:51) (cid:51) (cid:51) 𝑂 ( 𝑛 ) 𝑂 ( 𝑛 ) 𝑂 ( 𝑛 ) RandShare [126]
PVSS Async. no (cid:55) (cid:51) (cid:51) 𝑂 ( 𝑛 ) 𝑂 ( 𝑛 ) 𝑂 ( 𝑛 ) RandHound [126]
PVSS Sync. no 0.08% (cid:51) (cid:55) 𝑂 ( 𝑐 𝑛 ) (cid:5) 𝑂 ( 𝑐 𝑛 ) 𝑂 ( 𝑐 𝑛 ) RandHerd [126]
PVSS + CoSi Sync. yes ♯ (cid:51) (cid:51) 𝑂 ( 𝑐 log 𝑛 ) (cid:5) 𝑂 ( 𝑐 log 𝑛 ) 𝑂 ( ) Scrape [134]
PVSS Sync. no (cid:51) (cid:51) (cid:51) 𝑂 ( 𝑛 ) 𝑂 ( 𝑛 ) 𝑂 ( 𝑛 ) HydRand [133]
PVSS Sync. no (cid:51) (cid:37) (cid:51) (cid:51) 𝑂 ( 𝑛 ) 𝑂 ( 𝑛 ) 𝑂 ( 𝑛 ) Proof-of-work [1]
Hash Func. Sync. no (cid:51) (cid:37) (cid:55) 𝑂 ( 𝑛 ) very high ★ 𝑂 ( ) Proof-of-delay [150]
Hash Func. Sync. no (cid:51) (cid:51) (cid:51) 𝑂 ( 𝑛 ) very high ★ 𝑂 ( log Δ ) † Caucus [151]
Hash Func. Sync. no (cid:51) (cid:37) (cid:55) 𝑂 ( 𝑛 ) 𝑂 ( ) 𝑂 ( ) ♯ In Dfinity and RandHerd, nodes are devided into smaller groups, and within each of these groups a DKG protocol is required. * In Algorand and Ouroboros Praos, the approaches for generating randomness require additional communication and verification steps for theunderlying consensus protocols or the implementation of a bulletin board. Here we do not take the additional steps into consideration. (cid:5)
In RandHound and RandHerd, 𝑐 is a security parameter and depends on 𝑛 . If 𝑐 is constant, RandHound thereby reduces the asymptotic cost to 𝑂 ( 𝑛 ) and RandHerd further reduces the cost of producing successive beacon outputs to 𝑂 ( log 𝑛 ) per server. ★ In proof-of-work and proof-of-delay, the computational complexity is not dependent on the number of nodes 𝑛 . † In proof-of-delay, the verification is executed within a smart contract via an interactive challenge/response protocol which has logarithmic verificationcomplexity 𝑂 ( Δ ) in the security parameter Δ . § In RandRunner, only unpredictability is affected by network asynchrony while all other properties remain unchanged. After the network conditionsnormalize, unpredictability is restored and the recovery time increases linearly with the duration of the asynchronous period [153]. ♣ In RandRunner, if all nodes execute the protocol properly and the network is reliable, the communication complexity is 𝑂 ( 𝑛 ) . When concerningan adversarial leader, the communication complexity changes to 𝑂 ( 𝑛 ) (reliable broadcast) or 𝑂 ( 𝑛 log 𝑛 ) (gossip protocol). ‡ 𝑇 is the correct nodes? upper bound for the computation time of a VDF. .3.2. Randomness Properties As for randomness properties, “ (cid:51) ” in Tabel 3 denotes that protocols achieve corresponding propertiesunconditionally. “ (cid:55) ” means the protocol does not satisfy such properties.In regard to liveness property, Algorand and Dfinity consider failure probabilities of at most 10 − [72],[112], RandHound and RandHerd are 0.08% [126] while RandShare does not guarantee liveness under fullasynchrony since malicious nodes might never send messages.For unpredictability, “ (cid:37) ” denotes probabilistic guarantees for unpredictability, which quickly (exponen-tially in the waiting time) get stronger the longer a client waits after it commits to using a future protocoloutput [133]. In Algorand [72], Ouroboros Praos [112], PoW [1] and Caucus [151], the new randomnessdepends on the miner’s or the leader’s secret value. As long as malicious nodes mine a sequence of blocks orare selected repeatedly as leaders, prediction becomes possible. This problem can be solved by letting honestnodes participate in block production or be selected as leaders. Therefore, the probability of a successfulprediction decreases exponentially with the number of rounds. Moreover in HydRand [133], nodes are notallowed to be leaders again within 𝑓 rounds ( 𝑓 is the maximum number of byzantine parties) which onlyachieves unpredictability after 𝑓 + Complexity evaluation includes communication complexity (the overall bits transmitted by all nodesper round), computational complexity (the number of operations by one node per round), and verificationcomplexity (the number of operations by an external verifier per round).Obviously, in proof-of-work [1], proof-of-delay [150], and Caucus [151] protocols, a miner only has toperform one broadcast which leads to a communication complexity of 𝑂 ( 𝑛 ) . For Ouroboros Praos, com-munication complexity is not provided in the original work [112], here we refer to [133] which infers thatOuroboros Praos has a communication complexity in 𝑂 ( 𝑛 ) , because the protocol only provides guaranteesfor eventual consensus and is based upon many of the design principles of PoW blockchains. Other protocolslike Ouroboros [54], Scrape [134], and RandShare [126] are all based on the PVSS scheme, where each nodecommit to a secret value and broadcast a message of size 𝑂 ( 𝑛 ) to all other nodes, leading to a communicationcomplexity of 𝑂 ( 𝑛 ) . HydRand [133] reduces communication complexity to 𝑂 ( 𝑛 ) because only a single node(leader) has to perform the distribution of PVSS shares per round. RandRunner [153] achieves a communi-cation complexity of 𝑂 ( 𝑛 ) if all nodes follow the protocol and the network is reliable. When concerning anadversarial leader, it assumes two possible strategies for message dissemination, namely reliable broadcastand gossip protocol. The former means every honest node sends any valid message it received to all othernodes, resulting in a communication complexity of 𝑂 ( 𝑛 ) , while the latter is suitable for a large number ofnodes and the complexity is 𝑂 ( 𝑛 log 𝑛 ) .As for computational and verification complexity, proof-of-work [1] and proof-of-delay [150] achieve highcomputational complexity for the reason that both of them rely on solving cryptographic puzzles. TheVRF-based approaches such as Algorand [72], Ouroboros Praos [112], Caucus [151] (after the initial setup)as well as HE-Rand [93] are efficient, because they only require the verification of a VRF or hash preimage.The verification of RandRunner [153] only requires two hash functions and one verification algorithm with2 𝑡 as exponentiation ( 𝑇 = 𝑡 where 𝑇 is the time parameter of the VDF).To sum up, proof-of-work [1] and proof-of-delay [150] approaches are suitable for larger and dynamicsets of participants. RandRunner [153] is very resilient to temporary network delays or network breaks.Ouroboros [54], RandShare [126], Scrape [134] and HydRand [133] are more suitable for smaller groupsdue to their high communication complexity with the increasing number of nodes. Nguyen-Van et al. [93]use homomorphic encryption (ElGamal on elliptic curves) to encrypt shares, while they do not considera colluded user (“Requester”). Cachin et al. [143] come with a formal security proof in the asynchronous34etwork model, but it is based on elliptic curve pairings which are not yet well-established. Both RandHound[126] and RandHerd [126] divide all the participants into small groups. However, RandHound does not offerbias-resistance and RandHerd needs DKG protocol during setup and requires additional “view-change” [43]if the current leader fails to take adequate steps. Dfinity [90] provides strong bias-resistance but relieson DKG protocol during initialization which increases the communication complexity. Algorand [72] andOuroboros Praos [112] achieve better scalability while weakening bias-resistance. Caucus [151] can be easilydeployed in a smart contract, but it is easily biased. In the following, we analyze potential problems and future directions related to randomness genera-tion in sharding blockchains from the following three perspectives, i.e., security requirements, performanceimprovements and rigorous analysis.
As shown in Table 3, some of these schemes do not guarantee either the generation of perfectly unpre-dictable random values or that a value will be generated regardless of adversarial behavior. Specifically, anadversary might cause a liveness failure, try to bias the randomness, predict future random beacon outputsbefore the honest nodes get the values, or deceive a third party into accepting an invalid randomness. Be-sides, VDF-based approaches also have problems that their security depends on very accurate estimates ofthe average concrete complexity of certain computational processes [147] (i.e., the time delay parameter),which are hard to obtain in practice.Therefore, how to ensure the security guarantees (randomness properties mentioned in Section 5.1) stillneeds to be studied in the future.
Generally speaking, the more nodes participating in the consensus, the more secure a system will be.But on the other hand, the communication overhead also increases with the number of nodes. In Table 3,the methods that do have perfect security guarantees suffer from higher computational and communicationcomplexity, especially, some PVSS schemes (Ouroboros [54], Scrape [134]) have a complexity of 𝑂 ( 𝑛 ) .Moreover, the approaches based on threshold cryptographic schemes need to execute the DKG protocolduring the setup phase, which may increase the communication complexity. Besides, most approachesdo not support frequent changes within the nodes set. When some new nodes join the network, it takesadditional time and requires transferring more data than the original process.Consequently, on the premise of ensuring the availability of the sharding blockchains, how to reasonablydesign the randomness generation process, organize participation in node communication, balance scalability,and security requirements, and achieve efficient implementation of the protocol are issues that need to beresolved in the future. The rigorous analysis of a randomness generation protocol mainly includes formal definitions, preciseassumptions, and formal security proofs. Formal definitions mean the definitions of adversary model, net-work model, and randomness properties that a protocol satisfies, while precise assumptions refer to theassumptions of the underlying cryptographic schemes. Formal security proofs should be strictly logical,which prove that under certain definitions and assumptions, no adversary can successfully break the schemewith overwhelming probability.As we analyzed above, most approaches are under synchronous models that are relatively strong andmight be temporarily violated. For example, in HydRand, any leader which (temporarily) fails to deliverrequired messages is excluded from further participation, and the round duration parameter has to becarefully selected to avoid liveness failures [133]. Thus, how to address the current limitations should beconsidered in future works.Rand ∗ family [126], HydRand [133], and Scrape [134] are typical representatives of stand-alone protocols,which are specifically designed to generate randomness. Namely, these protocols are not used in isolation but35s building blocks of more complex systems. Therefore, the composability of these protocols is an importantissue. In particular, a UC secure protocol ensures that it can be used as a building block for more complexsystems while retaining its randomness properties, which is essential for randomness beacons.
6. Node Assignment
In this section, node assignment is analyzed. We first give the basic concepts of node assignment inSection 6.1. Then in Section 6.2, existing approaches are classified into binomial distribution and hyperge-ometric distribution. In addition, potential problems and future directions are discussed in Section 6.3.
The new nodes need to be randomly assigned to multiple shards. Otherwise, an adversary might gatherthe colluding nodes into a certain shard, thereby controlling the entire shard. In order to achieve the securityof the node assignment, randomness must be unpredictable, unbiased, and public verifiable. The definitionsof the specific properties of randomness is given in Section 5.1. The problems in the node assignmentprocess are as follows. First, it is necessary to ensure that the entire allocation process is random, thatis, the adversary cannot bias the allocation process. This requires that the random number is safe, anda pseudo-random number generator is used to generate a corresponding random number for each node.Second, the parameters need to be set reasonably to ensure that the number of honest nodes in each groupmeets the standard. This requires the use of a certain mathematical model to strictly analyze the finaldistribution result when the node is a non-infinite pool.
Let 𝑛 denote the total number of nodes participating in a protocol. Let 𝑚 represent the number ofshards, and let 𝑢 denote the number of nodes in a single shard. The node assignment process is regarded as a random sampling problem. Under the following assumption,the binomial distribution can be used.The nodes before the distribution process are assumed to form an infinite pool. In other words, each timea node is selected from the infinite pool, the probability that the node being honest or malicious remainsconstant.The probability here refers to an adversary’s computational power, which is denoted by 𝜌 . Assume thatthe target honest fraction in a committee is 𝑄 , which might be 2 / /
2. Let 𝑋 denote the number oftimes that picking a malicious node. The probability that an adversary’s proportion in a selected committeeis exactly a certain value, e.g., 1 − 𝑄 , is calculated through the binomial distribution as in Equation 1.Pr [ 𝑋 = 𝑢 ( − 𝑄 )] = (cid:18) 𝑢𝑢 ( − 𝑄 ) (cid:19) 𝜌 𝑢 ( − 𝑄 ) ( − 𝜌 ) 𝑢𝑄 (1)When a selected committee is malicious, we say a distribution is failed, i.e., the fraction of maliciousnodes exceeds a predefined target value 1 − 𝑄 . So the cumulative binomial distribution could be adoptedto calculate the failure probability, where 𝑋 is supposed to be greater than the value 𝑢 ( − 𝑄 ) . Hence, thefailure probability could be calculated as shown in Equation 2.Pr [ 𝑋 ≥ (cid:100) 𝑢 ( − 𝑄 )(cid:101)] = 𝑢 ∑︁ 𝑥 = (cid:100) 𝑢 ( − 𝑄 ) (cid:101) (cid:18) 𝑢𝑥 (cid:19) 𝜌 𝑥 ( − 𝜌 ) 𝑢 − 𝑥 (2)Omniledger [58] employs the cumulative binomial distribution described above.36 .2.2. Hypergeometric Distribution The infinite pool assumption in binomial distribution means the member selection process does notinfluence the probability of being honest or malicious in the selected node. The selection is done with nodereplacement, i.e., the selected node has to be replaced back to the pool. On the contrary, hypergeometricdistribution does not assume an infinite pool, which means the distribution is done without replacement.The failure probability is calculated by the cumulative hypergeometric distribution, as shown in Equa-tion 3. Pr [ 𝑋 ≥ (cid:100)( − 𝑄 ) 𝑢 (cid:101)] = 𝑢 ∑︁ 𝑥 = (cid:100) 𝑢 ( − 𝑄 ) (cid:101) (cid:0) 𝜌𝑛𝑥 (cid:1) (cid:0) 𝑛 ( − 𝜌 ) 𝑢 − 𝑥 (cid:1)(cid:0) 𝑛𝑢 (cid:1) (3)RapidChain [59] and SGX sharding [95] analyze the epoch security utilizing the cumulative hypergeo-metric distribution. Hafid et al. [96] carry out a probabilistic security analysis of sharding blockchains usingtail inequalities such as Hoeffding inequality [159] to approximate the upper bound of the failure probabilityfor each epoch. In fact, both binomial distribution and hypergeometric distribution are based on random allocation ofparticipating nodes, while some existing schemes use special node distribution rules.In PolyShard [160], nodes store and compute on a coded shard of the same size that is generated bylinearly mixing uncoded shards. PolyShard uses the Lagrange Coded Computing [161] technology to codeshards.
The processes of node selection and node assignment are connected together, so we analyze the problemsin the whole procedures.As shown in Figure 10, A is used to denote new nodes, i.e., all nodes that want to participate in theprotocol. B represents selected new nodes and C refers to confirmed committees. From A to B , there mustbe a mechanism such as PoW to defend against the Sybil attacks [83]. Furthermore, from B to C , a securerandomness is in need to assign selected nodes into multiple committees. Figure 10: The process of node selection and node assignment. A to B is ignored In Omniledger [58] and RapidChain [59], the protocol contains the steps from A to B and then from B to C . However, they do not consider the changes in the adversarial proportion from A to B . In fact, theadversarial proportion in B is larger than that of A due to several reasons. First, an adversary might havean advantage in message transfer so that he could start mining in advance of honest nodes. Second, if aleader in a reference committee is malicious, then he could launch a node censorship attack as described inSection 4.3 to increase the proportion of nodes under his control. Assume that the computational power37roportion of an adversary is 𝜌 , then the proportion of malicious nodes in B is greater than 𝜌 with a highprobability. The analysis in Omniledger and RapidChain still regards 𝜌 as an initial input, which wouldlead to false results after random allocation. As a result, the confirmed committees in C could be malicious. The node assignment process is to randomly allocate selected nodes B , while the number of nodes in B islimited. Whenever a node is selected, the proportion of malicious nodes in B changes. Therefore, assumingan infinite pool is inaccurate, the probability obtained in this way will have a deviation. When computing the failure probability of an epoch in RapidChain, the failure rates of all committeesis calculated through Equation 3, which is imprecise. In fact, Equation 3 could only be used to computethe failure rate of the first committee. However, after the current committee is confirmed, the subsequentselection of committees will be affected by the current one, that is to say, the parameters of the cumulativehypergeometric distribution have been changed at this time. Therefore, the calculation of the epoch failureprobability is also inaccurate. The impact of the first committee on subsequent allocation parameters is notconsidered.
7. Intra-Shard Consensus
As described in the modular design in Section 3, the intra-shard consensus is the key component forevery sharding blockchain. In this section, we research intra-shard consensus protocols. In Section 7.1, basicconcepts of intra-shard consensus protocols are given. Based on this, we divide sharding blockchains intoinstant and eventual sharding blockchains according to their intra-shard consensus and give their definitions,respectively. Section 7.2 introduces the state machine replication algorithms that may be used in shardingblockchains from the aspects of different network models. Finally, Section 7.3 summarizes potential problemsthat might occur in intra-shard consensus protocols.
The main purpose of the intra-shard consensus is to efficiently process the transactions within a shard.In a sharding blockchain system, the intra-shard consensus needs to cooperate with other shards to commitcross-shard transactions. This requires the intra-shard consensus algorithm to provide availability certificatesof the relevant transaction inputs, that is, to generate proofs of inputs. The proofs appear in the form ofsignatures. In addition, in some sharding blockchains that adopt reference committees, the intra-shardconsensus is also used to confirm the list of new committee members. The intra-shard consensus algorithmgreatly affects the efficiency of transaction processing.Several issues need to be considered for intra-shard consensus. First, the scalability of the intra-shardconsensus algorithm should be taken into account. The scalability is related to the communication andcomputational complexity inside the shard. The transaction processing capabilities might sharply decreaseas the number of nodes in the shard increases. Second, in order to meet the special needs of the shardingblockchain scenario, the intra-shard consensus algorithm needs to handle different types of proposals. Third,the relationship between the intra-shard and the entire network transmission model needs to be taken intoaccount.Intra-shard consensus algorithms could be divided into two categories, i.e., strong consistency consensusalgorithms and weak consistency consensus algorithms. In a weak consistent (ref. Definition 16) blockchain,a block producer is a single node, determined by PoW or PoS within each shard. We call such blockchainsas eventual sharding blockchains (ref. Definition 29). In a strong consistent blockchain (ref. Definition 20),each shard runs a committee, and the committee acts as the block producer, running a distributed consensusalgorithm, e.g., PBFT, to confirm transactions and generate blocks. We call such blockchains as instantsharding blockchains (ref. Definition 28). 38 .2. Existing Approaches
In the following, we discuss the algorithms that could be used as intra-shard consensus in shardingblockchains, including strong consistent and weak consistent algorithms.
The intra-shard consensus algorithms for instant sharding blockchains are usually some classical dis-tributed consensus algorithms or some adaptions of them which realize strong consistency.In classical distributed consensus algorithms, a group of nodes in a permissioned network realizes statemachine replication (ref. Definition 21), achieving consistency and liveness.In general, classical distributed consensus algorithms have different assumptions on the situation of nodesin the network, such as whether crash or Byzantine nodes (ref. Definition 22) exist. It is usually consideredthat the behaviors of Byzantine nodes include those of crash nodes, so protocols that tolerate the Byzantinenodes are more applicable and robust. In the context of blockchain, various BFT (ref. Definition 23)protocols have been proposed. We mainly introduce the research on BFT below since BFT protocols couldbe well combined with blockchain to form a so-called hybrid consensus.According to the network model assumptions, classical distributed consensus algorithms could be dividedinto the following three categories: classical distributed consensus algorithms in synchronous networks,asynchronous networks, and partially synchronous networks.
Synchronous networks.
In the following, we first introduce some representative distributed consensus algo-rithms under synchronous networks, then we describe their applications to sharding blockchains. (1) Distributed consensus algorithms:
As described in Definition 1, in a synchronous network, the mes-sages sent by honest nodes in a certain round must reach each other before the next round. As a result,the message transmission delay Δ is used as a parameter in related protocols, which simplifies the protocoldesign to some extent.The Byzantine quorum system [97] first proposes the concept of a “quorum”, which could be seen asthe minimum number of votes required for honest nodes to agree on a proposed value in a voting round.The quorum is to prevent a malicious leader from equivocating, that is, sending different proposals todifferent honest members in the same round. The concept of a quorum is applicable in both synchronousand partially synchronous networks. Specifically, in a partially synchronous network where the adversarymodel is 𝑢 = 𝑓 +
1, a quorum is set to 2 𝑓 +
1, which means in a voting round, an honest member considersthis voting round to be successful only after collecting at least 2 𝑓 + 𝑝 and 𝑝 (cid:48) to different members, itis proved by contradiction that 𝑝 and 𝑝 (cid:48) cannot be committed at the same time. Assume that both 𝑝 and 𝑝 (cid:48) get 2 𝑓 + 𝑢 members. Then there are at least 2 𝑓 + + 𝑓 + − ( 𝑓 + ) = 𝑓 + 𝑝 and 𝑝 (cid:48) , which is contradictory to the assumption that there are only 𝑓 malicious members, sincehonest members will only vote for one proposal value in a round.Sync HotStuff [98] assumes a synchronous network and utilizes the pipeline technology to improve pro-posals. Different from HotStuff, Sync HotStuff adopts a two-phase leader-based method to process proposals.The transaction confirmation delay is declared as 2 Δ in a steady state where Δ denotes the upper bound ofmessage transmission delay.Other distributed consensus algorithms under the synchronous network model include XFT [162], prac-tical synchronous Byzantine consensus [163], Ouroboros-BFT [164], Flexible BFT [165], Hybrid BFT [81],PiLi [166], etc. These schemes could be applied to sharding blockchains by adding specific interfaces. (2) Combined With Sharding Blockchains: Note that in a sharding blockchain, the message transmissionmodel of the entire network might be different from that inside a shard. That is to say, the network modelwithin the shards could be a synchronous network, while the entire network is synchronous or partiallysynchronous.RapidChain [59] uses a variant of the practical synchronous Byzantine consensus proposed in [163].The adversary model is still 𝑢 = 𝑓 +
1, namely, it could tolerant nearly 1 / propose , echo , pending , and accept . A leader broadcasts the message with its39ash value in the propose round. Then every node receiving the message broadcasts its hash value amongthe network in the echo round, to ensure every honest node obtains all messages sent by the leader. If amalicious leader sends more than one message to different nodes in the propose round, then honest nodeswill detect it during the echo round, mark the conflicting messages with a pending tag, and broadcast thepending messages. In this case, the malicious leader will be replaced. In the normal case, honest nodes thathave received 𝑓 + / Δ time in each round of communication. This is also one ofthe differences between synchronous and partially synchronous network model protocols. The parameter ofthe upper limit of network delay Δ could be directly used in the synchronous network model protocols, butnot in the partially synchronous and asynchronous network model protocols. Since every node needs to waitfor a fixed time in each round, the transaction confirmation time of the protocol is not related to the actualdelay of the network, so that the property of responsiveness is not achieved. Meanwhile, the synchronousnetwork model puts high requirements on the network status and is not particularly applicable in reality. Partially synchronous networks.
The partially synchronous network is a model adopted by most blockchainsystems, and it is also a model closer to the network in reality. Consequently, there are many studies inthis field. In the following, we first describe several typical schemes, then we introduce the combination ofdistributed consensus algorithms and sharding blockchains. (1) Distributed consensus algorithms:
The distributed consensus algorithms in partially synchronousnetworks are represented by Paxos, PBFT and its improvements. (i) Paxos
Paxos [100] is designed for database maintenance in distributed systems. In Paxos, a primary nodesends a prepare message to more than 1 / 𝑓 crash nodes in the 𝑢 = 𝑓 + (ii) PBFT and its improvements In most sharding blockchains or committee-based blockchains, the consensus algorithm within a com-mittee is the PBFT algorithm or its adapted version, so the PBFT algorithm is of vital importance. Hence,in the following, we introduce in detail the basic process of the PBFT algorithm and some recent researchon its improvement.PBFT utilizes a similar name to distinguish nodes as Paxos, i.e., primary and backup nodes. In apartially synchronous network, PBFT assumes the 𝑢 = 𝑓 + propose phase, a client (user) uploads a proposal 𝑝 to all nodes. Second, in the pre-prepare phase, the primary node constructs a pre-prepare message ( pre-prepare , 𝐻 ( 𝑝 ) , 𝑠, 𝑣 ) , where 𝐻 (·) is a one-wayhash function, 𝑠 denotes the sequence number, and 𝑣 represents the view number. The primary node sendsthe pre-prepare message to all replicas. Third, in the prepare phase, every replica node confirms that forthe same ( 𝑣, 𝑠 ) , no conflicting preparation message has been received, then broadcasts the prepare message ( prepare , 𝐻 ( 𝑝 ) , 𝑠, 𝑣 ) . Fourth, in the commit phase, after receiving 2 𝑓 + 𝑝 as prepared and broadcasts a commit message ( commit , 𝐻 ( 𝑝 ) , 𝑠, 𝑣 ) . Fifth, in40 igure 11: Process of PBFT algorithm the reply phase, after receiving 2 𝑓 + 𝑝 ascommitted, and returns the committed proposal with its signatures to the client [29].If a primary node behaves maliciously or does not respond, PBFT relies on a view-change mechanism tochange a primary node. A checkpoint mechanism is designed to assist the view-change, where the maximumsequence number of all committed proposals is regarded as a stable checkpoint. The specific steps of aview-change mechanism are as follows. First, in the view-change message broadcast phase, node 𝑖 broadcastsa view-change message vc 𝑖 : ( view-change , 𝑣 + , 𝑆 ∗ , 𝐶, 𝑈, 𝑖 ) where 𝑣 is the view number, 𝑆 ∗ stands for thecurrent stable checkpoint number, 𝐶 denotes the set of 2 𝑓 + 𝑆 ∗ , and 𝑈 is a setthat contains the prepared messages whose serial number is greater than 𝑆 ∗ . Second, in the view-changeacknowledgment phase, a replica verifies the view-change message and constructs a corresponding view-change acknowledgment message vca 𝑖 : ( view-change-ack , 𝑣 + , 𝑖, 𝑗, 𝐻 ( vc 𝑗 )) , where 𝑖 is the current replicanode, 𝑗 is the node that sends the view-change message vc 𝑗 , and 𝐻 ( vc 𝑗 ) is the hash digest of the view-change message. The replica sends vca 𝑖 directly to the new primary node of view 𝑣 +
1. Third, in the new-view broadcast phase, for each view-change message, when the primary node collects 2 𝑓 − 𝑗 , then vc 𝑗 is valid and put into the set 𝑆 . The new primary node constructsa new-view message nv : ( new-view , 𝑣 + , 𝑆, 𝑈 ∗ ) where 𝑈 ∗ includes the current stable checkpoint and a pre-prepare message with the smallest sequence number after the stable checkpoint. The node verifies thevalidity of the messages in 𝑆 , updates its local states according to 𝑈 ∗ , and enter view 𝑣 +
1. For more details,readers may refer to [43].PBFT achieves the consistency and liveness properties of the state machine replication, and the commu-nication complexity is 𝑂 ( 𝑛 ) in normal operations. In a view-change process, the communication complexityis 𝑂 ( 𝑛 ) . PBFT needs to be improved to make it better applied to sharding blockchains.The HotStuff[101] algorithm is proposed to improve PBFT, making the BFT algorithm and blockchainachieve a better combination. HotStuff adopts a partially synchronous network model, and the adversarymodel is 𝑢 = 𝑓 +
1. HotStuff mainly has three adaptions. First, it uses the pipeline technology to processproposals, that is, the message in a round contains a quorum certificate of the previous round and a newproposal. Second, it adopts the BLS threshold signature to aggregate 2 𝑓 + 𝑝 𝑎 refers to the proposal of node 𝑎 in the first round. The message of the first roundis denoted as 𝑚 . 𝑚 is broadcast by node 𝑎 . Then the other nodes verify 𝑚 and vote for it. Node 𝑏 actsas a leader and collects valid votes. When the number of valid votes reaches 2 𝑓 +
1, node 𝑏 reconstructs aBLS threshold signature using the 2 𝑓 + ( 𝑚 ) . Then node 𝑏 constructs a message 𝑚 by combining QC ( 𝑚 ) and a new proposal 𝑝 𝑏 , and broadcasts 𝑚 to other nodes.At this time, node 𝑎, 𝑐 and 𝑑 act as replicas. After receiving 𝑚 , they first verify the validity of QC ( 𝑚 ) 𝑝 𝑏 , and then vote to 𝑚 if the verification is passed. The subsequent operations are similar. Note thatthe proposal is allowed to be ⊥ to ensure the liveness property of the protocol as shown in round 4, sinceit is necessary to consider the situation where there might be no transaction being uploaded. The linearview-change in HotStuff refers to the message sent by the leader since only a single threshold signature isin need during view-change instead of 2 𝑓 + Figure 12: Process of HotStuff algorithm
In addition to PBFT and HotStuff, there are other Byzantine fault tolerant algorithms in partially synch-ronous networks, such as scalable Byzantine fault tolerance [171] and PaLa [103]. They could be regardedas independent consensus algorithms or could be applied to sharding blockchains after some improvements. (2) Combined with sharding blockchains:
The research on the combination of Paxos and blockchains[167, 168] is little. In contrast, most instant sharding blockchains adopt the PBFT algorithm or some variantsof it as the intra-shard consensus algorithm. ELASTICO [31] uses PBFT directly in every committee toprocess transactions. However, the committee of each shard in ELASTICO cannot finally complete thecommitment of transactions. They have to send the signatures to the final committee, who will run PBFTto commit transactions. Therefore, ELASTICO cannot handle cross-shard transactions, and its transactionprocessing efficiency is low.Omniledger designs Omnicon based on Byzcoin [56], which makes some adaptions of PBFT. We firstintroduce Byzcoin and then introduce the adaptions of Omnicon.In Byzcoin, MAC is replaced by digital signatures, and they use a tree communication model and theCoSi [148] protocol, a scalable collective signing, to cut down message length. CoSi combines the well-knownSchnorr multi-signatures [172] with communication trees, and generates a single signature on a message froma group of nodes. CoSi consists of four phases, namely, announcement, commitment, challenge, and response.Byzcoin combines CoSi with PBFT’s voting mechanism. A leader’s announcement is the same as that of thepre-prepare phase in PBFT. The commitment of other nodes implements the prepare votes. Then the leadercollects enough votes and initiates a challenge phase, and the other nodes respond to it, which implementsthe commit phase in PBFT. Finally, a single signature from 2 𝑓 + 𝑂 ( ) compared with 𝑂 ( 𝑛 ) in PBFT.In Omnicon, the authors point out that the CoSi used in Byzcoin is susceptible to faults since the depthof the communication tree is much too large. So they change the message propagation mechanism. In eachshard of Omnicon, nodes are divided into several groups based on some generated randomness, to decreasethe depth of the communication tree. A shard leader requires signatures from several group leaders. Eachgroup leader is responsible for collecting messages inside its corresponding group. If a group leader does notrespond in a certain time, the shard leader will randomly choose another node in the group as a leader. Ifa shard leader is offline or behaves maliciously, Omnicon still relies on a view-change launched by all shardmembers to change the leader.ZILLIQA [91] improves the efficiency of PBFT by using the EC-Schnorr multi-signature protocol [172].The idea is similar to Byzcoin. In the process of signing, signers are required to maintain a bitmap whichindicates who has signed the message. The bitmap is first built by a leader and could be used as an index42or a verifier to verify the signature efficiently.Chainspace [102] uses MOD-SMART implementation [173] of PBFT as the intra-shard consensus. MOD-SMART makes adaptions to PBFT, where reliable broadcast is substituted for validated and provableconsensus. Asynchronous networks.
Next, we introduce the distributed consensus algorithms in asynchronous networks.Fischer, Lynch, and Paterson [174] propose the FLP impossible theorem. They point out that in areliable asynchronous network that allows node failure, there is no deterministic consensus algorithm thatcould solve the consensus problem.Honey Badger BFT [99] is a well-known asynchronous BFT algorithm, so we take it as a representativeand introduce its operations here. The adversary model of Honey Badger BFT is 𝑢 = 𝑓 + 𝑢 nodestry to reach agreement on 𝐵 transactions in a round. The asynchronous BFT algorithms mainly rely onreliable broadcast (RBC) and asynchronous binary agreement (ABA) as building blocks to realize consensus.The basic concepts of Honey Badger BFT are as follows.First, in the transactions collection phase, the 𝑢 participating nodes all collect transactions submittedby users. Second, in the transaction threshold encryption phase, each node uses the threshold encryptionfunction to encrypt 𝐵 / 𝑢 of its transactions collected in every single round. Third, in the RBC broadcastphase, each node relies on the RBC to broadcast and collect messages. RBC contains two rounds, namelythe echo and ready round. When a node receives a certain number of ready message on a value, it believesthat the value arrives at all honest nodes. Fourth, in the ABA consensus phase, a leader is responsible forcollecting all encrypted values sent by other nodes and initiating the ABA algorithm on them. In the ABAalgorithm, nodes decide on whether the total transaction set is valid through multiple voting rounds. Fifth,in the transaction threshold decryption phase, if the encrypted transaction set is valid, then each node runsthe threshold decryption algorithm. As long as the number of nodes who complete the decryption exceedsthe predetermined threshold, i.e., 𝑓 +
1, the total transaction set could be decrypted and the transactionconfirmation is completed. At the same time, the transaction censorship attack [175, 99] is prevented sincethe transactions appear in the form of ciphertext when a leader proposes the set. For more details, pleaserefer to [99].There are some other distributed consensus algorithms under the asynchronous network model, such as[176] [177], asynchronous binary Byzantine agreement [143], MinBFT [178], validated asynchronous Byzan-tine agreement [179], BEAT [180], and Dumbo-MVBA [181, 182]. These algorithms mainly rely on reliablebroadcast and binary Byzantine agreement to reach a consensus within the committee.As far as we know, there is no sharding blockchain that uses a distributed consensus algorithm in anasynchronous network as the intra-committee consensus. The reason might be that the confirmation ofcross-shard transactions relies on the liveness of the consensus algorithm in each shard. When a shardingblockchain processes cross-shard transactions, multiple shards need to cooperate and respond within a certaintime, while asynchronous distributed consensus algorithms generally sacrifice liveness to ensure security whena network partition occurs.
Eventual sharding blockchains still use PoW, PoS, or other weak consistency methods to generate blocksin each shard. There is no committee in each shard. If PoW is used to generate blocks, nodes in a shardneed to find the pre-image of the hash function that meets the specific requirements. If PoS is adopted, anode needs to check whether it becomes a block producer through some mechanisms like VRF according tothe stake it holds.Monoxide [60] proposes chu-ko-nu mining to realize intra-shard consensus with PoW. Chu-ko-nu miningis mainly designed to defend against the 1% attack where an adversary focuses his computational power onone single shard. In Monoxide, miners are required to mine on multiple blockchains using one hash function.The 𝑚 block headers at the end of 𝑚 different blockchains form a Merkle tree, where the root value ofthe Merkle tree and a 𝑛𝑜𝑛𝑐𝑒 are used as inputs of the hash function. In this way, the 1% attack could beprevented since the inputs of the hash function used in mining are related to 𝑚 blockchains and change inreal-time. However, since the 𝑚 blockchains are constantly extended and new blocks are generated, miners43eed to collect and verify all newly generated blocks of 𝑚 blockchains continuously to ensure their mininginputs are valid. As a result, miners have to collect and verify all transactions in the network, so in essence,Monoxide does not achieve scalability [108].Parallel Chains [89] combines VRF and PoS to generate blocks in each shard. In each round, each nodemight become the block producer of one or more shards out of the 𝑚 shards. Each node uses VRF togenerate 𝑚 outputs and corresponding proofs. If an output is lower than the specific value determined bythe protocol, then the node becomes the block producer of the shard corresponding to the output. At thistime, the node packages the transactions belonging to the shard to generate a block and broadcasts theblock with the output and proof of VRF on the entire network. With the emergence and continuous development of the blockchain technology, the classic state machinereplication algorithms have been continuously researched and improved, since it matches well in the contextof blockchain. However, there are still some problems hindering its further application. We summarize theseissues from the perspective of instant sharding blockchains and eventual sharding blockchains as follows,and point out future research directions.
The possible problems and research directions of instant sharding blockchains mainly include the followingpoints. • Reducing the communication complexity among shard members.
The distributed consensus algorithmsinside a shard generally rely on multiple voting rounds to reach an agreement. In the voting process,if each member broadcasts its own signature, collects and verifies the signatures of all other members,when the number of members increases, the broadcast and collection of signatures will cost huge com-munication bandwidth and time, thereby reducing the efficiency of the entire protocol. In particular, ina permissionless sharding blockchain, in order to ensure that there are sufficient honest nodes in a shard,the number of shard members needs to reach a certain security threshold. In this case, how to reduce thecommunication complexity inside a shard and improve the processing efficiency is a problem that needsto be solved. • Malicious committee detection and recovery.
In an instant sharding blockchain, multiple committees rundistributed consensus algorithms. Although most sharding blockchains are designed with a series of as-sumptions and analyses to ensure that each committee is honest with a high probability, it is still possiblefor a certain committee to be controlled by an adversary. In this case, how to detect malicious commit-tees through other honest committees and design a specific mechanism to restore or replace maliciouscommittees with honest committees is one of the future research directions. • Efficient view-change mechanism inside a committee.
When a view-change occurs in a shard, a new leaderis required to replace the old leader, and the views of all members in the committee are unified. In thisprocess, how to reduce the communication complexity among members is a problem that needs to besolved. In addition, in a sharding blockchain, a leader not only serves as the message center inside ashard but also as a coordinator to transfer messages among different shards. Consequently, the leaderhas a higher overhead. How to share the burden of the leader and how to choose a new leader fairly andreasonably is one of the future research directions. • Better combination with sharding blockchains.
In sharding blockchains, the intra-shard consensus is notjust used to process transactions. In most cases, intra-shard consensus needs to handle cross-shard trans-actions. The processing of cross-shard transactions requires multiple shards to run multiple rounds ofintra-shard consensus, and the target of the consensus is not just a simple transaction but might be aninput of a transaction, to determine whether an input is available. In this way, there may be multipletypes of inputs for the intra-shard consensus, and the rules for judging whether the inputs of differenttypes are valid are also different. Therefore, the details of combining intra-shard consensus with the44ntire sharding blockchain protocol need to be designed more specifically, so as to process transactions insharding blockchains more efficiently.
In eventual sharding blockchains, there are mainly two potential problems that need to be handled. • The 1% attack problem.
Taking PoW-based sharding blockchains as an example, an adversary could focushis computational power on a single shard to mine. If the mining process of each blockchain in each shardis independent, then for each blockchain, it is usually required that the computational power controlled byan adversary cannot exceed 51% (if not consider selfish mining attacks). Assume that there are 𝑚 shards,that is, 𝑚 parallel blockchains, if the adversary concentrates its computational power on one of the shards,he only needs 51% / 𝑚 of the total computational power to fully control the blockchain. When 𝑚 is largeenough, 51% / 𝑚 is approximately equal to 1%, so an adversary only needs 1% of the computational powerto control the shard. In this case, the blocks in this shard will all be generated by the adversary, whichmeans that the adversary has full control over the current shard. The adversary could launch the double-spending attack, thereby destroying the security of the entire system. Therefore, in eventual shardingblockchains, how to prevent the 1% attack while ensuring the computing, storage, and communicationsharding (described in Section 2.1) simultaneously is one of the future research directions. • Complicated cross-shard transaction processing.
Due to the weak consistency property of eventual shardingblockchains, the blocks generated in a shard could not be confirmed as stable instantly. In general, a blockhas to reach a certain depth to be considered stable and valid. In other words, a certain number of blocksat the end of a blockchain must be truncated to confirm valid transactions. The number of blocks thatis removed is related to the system security parameter such as 6 in Bitcoin. As a result, cross-shardtransaction processing is difficult in eventual sharding blockchains. We discuss this in detail in Section 8.
8. Cross-Shard Transaction Processing
In this section, we analyze cross-shard transaction processing in sharding blockchains. In Section 8.1,basic concepts are given. In Section 8.2, we classify existing approaches regarding cross-shard transactionprocessing into three categories, namely, two-phase commit (2PC) based, transaction split based, and relaytransaction based solutions. For each type of processing method, its basic process is summarized and typicalschemes are discussed. Section 8.3 provides potential problems and future research directions.
In sharding blockchains, the probability for a transaction to be a cross-shard one is extremely high. Theprobability increases as the number of shards grows. As computed in RapidChain [59], when the numberof shards is 16, the proportion of cross-shard transactions is about 99 . .2. Existing Approaches The approaches to process cross-shard transactions could be divided according to whether there is acommittee in each shard, i.e., whether the blockchain is an instant sharding blockchain or an eventualone. In instant sharding blockchains, the most common methods to process cross-shard transactions aretwo-phase commit based and transaction split solutions. Relay transaction based solutions are designed foreventual sharding blockchains.
Most cross-shard transaction processing approaches are designed on top of the two-phase commit protocolwhich contains a prepare phase and a commit phase. In a 2PC protocol, there is a coordinator whois responsible for collecting availability certificates of inputs and transmitting them among the relatedparticipating shards. To formalize the process of 2PC based solutions, we give the definition of an availabilitycertificate.
Definition 32 (Availability Certificate).
An availability certificate refers to a proof in the prepare phaseof 2PC based cross-shard transaction processing methods that each shard provides for a transaction input,to prove that the input is available or unavailable.
In the prepare phase, a coordinator collects certificates to prove that the inputs of a transaction areavailable from different shards. Such a proof is usually generated by shard members through running BFTto reach an agreement on if the inputs are available. So the proof might be some signatures or a singleaggregated signature. Meanwhile, the inputs should be locked to prevent themselves from being spent byother transactions.Then in the commit phase, the coordinator sends all availability certificates of inputs to all related shards,including input and output shards. If all inputs are available, then the transaction is regarded as valid andcommitted in related shards. The inputs should be spent and outputs should be created. Else, if at leastone input is unavailable (locked or already spent), the transaction is invalid. The previously locked inputsshould be unlocked.According to the role of a coordinator, we divide current cross-shard processing methods into client-drivenbasic 2PC and shard-driven basic 2PC.
Client-driven 2PC.
The basic procedure is shown in Fig. 13. A client is responsible for collecting proofs inthe prepare phase and transmitting them to related shards in the commit phase.
Figure 13: The flowchart of client-driven basic 2PC.
Omniledger [58] adopts the client-driven 2PC methods to process cross-shard transactions. In the preparephase, an availability certificate is named as a proof-of-acceptance or a proof-of-rejection. A proof-of-acceptance is generated by a leader of an input shard committee, i.e., a leader signs on an input, to provethat the input is ready to be spent. 46 hard-driven 2PC.
In a shard-driven 2PC protocol, one or more shards play the role of coordinators. In theprepare phase, the availability certificates of inputs are generated by input shards. In the commit phase,a valid transaction will be accepted in all input and output shards. The confirmation information of thetransaction will be sent to the client. Compared with client-driven 2PC, the burden on a client is released.The client just submits a transaction and waits for the response.The flowchart is shown in Fig. 14. Note that in Fig. 14(a), all input shards act as the coordinators, whichmeans the availability certificates are transferred by the input shard directly. In Fig. 14(b), an output shardplays the role of a coordinator, collects the availability certificates, and forwards them to relative shards.We argue that in normal cases, the communication complexity of the two methods above is identical sincethe messages are simply collected and forwarded, without being aggregated, so the total number of messagesremains unchanged. (a) All input shards as the coordinators. (b) An output shard as the coordinator.Figure 14: The flowchart of shard-driven basic 2PC.
Chainspace [102], RSTBP [183] and FleetChain [184] adopt the shard-driven 2PC methods to handlecross-shard transactions. The coordinators in Chainspace are the input shards. The availability certificatesare produced by input shard committees running a BFT algorithm and broadcast among all participatingshards, including input and output shards.
The transaction split based approach is proposed in RapidChain [59] to handle cross-shard transactionsby splitting a multi-input multi-output transaction into multiple single-input single-output transactions. Asimple example is shown in Fig. 15.For a transaction tx with two inputs 𝐼 and 𝐼 of shard 𝐶 𝑖𝑛 and 𝐶 𝑖𝑛 , respectively, and one output 𝑂 belonging to shard 𝐶 𝑜𝑢𝑡 , a client submits tx directly to the committee members of 𝐶 𝑜𝑢𝑡 . Then 𝐶 𝑜𝑢𝑡 splits tx into three following transactions, tx : 𝐼 → 𝐼 (cid:48) , tx : 𝐼 → 𝐼 (cid:48) and tx : 𝐼 (cid:48) + 𝐼 (cid:48) → 𝑂 where 𝐼 (cid:48) and 𝐼 (cid:48) belongto 𝐶 𝑜𝑢𝑡 . If tx and tx are committed by 𝐶 𝑖𝑛 and 𝐶 𝑖𝑛 respectively, then tx will be executed successfullyby 𝐶 𝑜𝑢𝑡 and the cross-shard fund transfer process is completed. If there are more inputs for a cross-shardtransaction, then each input corresponds to a new transaction.The transaction split based approach simplifies the processing of cross-shard transactions to some extent,while it introduces many new problems, which we will analyze in detail in the next section. Relay transaction based approaches are usually adopted in eventual sharding blockchains. In eventualsharding blockchains, there is no BFT running in each committee, so transactions or availability certificateswill not be confirmed immediately. As a result, 2PC based cross-shard transaction processing methods couldnot be used.In eventual sharding blockchains, a transaction is considered as committed after its block reaches acertain depth of the blockchain such as 6 blocks in Bitcoin. The number of blocks (usually denoted by 𝜆 ) isusually related to system security. Consequently, in eventual sharding blockchains, the essence of processing47ross-shard transactions is to ensure that the output shard does not treat the transaction as valid until thetransaction in all related input shards is completely committed, i.e., reaches sufficient depth.The basic steps of relay transaction based solutions are as follows. First, the miners of the input shardcollect the cross-shard transaction tx , that is, an account 𝐴 wants to pay an account 𝐵 𝑦 amount of fund.A miner verifies whether the balance of account 𝐴 is greater than 𝑦 , and if it is, tx is regarded as a legaltransaction. Second, a miner finds a PoW solution, constructs a block containing tx , and broadcasts theblock. Other miners verify the block, generate a corresponding relay transaction 𝜓 , and send it to the minersof the corresponding output shard. Third, the miners of the output shard receive the relay transaction andverify the legitimacy of it, that is, 𝜓 has reached the block depth of 𝜆 in the input shard. Fourth, a miner ofthe output shard finds a PoW solution, adds valid relay transactions including 𝜓 to the block, and broadcastsit. When the block containing the relay transaction 𝜓 reaches a 𝜆 depth in the output shard, which is calleda 𝜆 -confirmation, it is determined that the 𝑦 amount of money could be spent by the account 𝐵 .Monoxide [60] adopts the relay transaction based solution to process cross-shard transactions. Meanwhile,the account model is utilized in their system. RChain [185] also utilizes a similar method to transfer valueacross shards. Buterin [186] also proposes a cross-shard contract yanking method to deal with contractsthat are related to multiple shards. The essence of the method is to use the relay transaction. Ostraka [187]adopts a node differential environment model, i.e., each node’s ability to process transactions is different. Next, we analyze the possible problems of each processing approach for cross-shard transactions andpoint out future research directions.
We analyze potential problems in 2PC based approaches from the following two aspects, i.e., client-driven2PC and shard-driven 2PC.
Client-driven 2PC.
Possible problems in client-driven 2PC based approaches are as follows. • Malicious behaviors of the leader.
In the prepare phase, if an availability certificate, i.e., a proof of aninput, is generated by a single leader like in Omniledger [58] instead of a committee running BFT, then amalicious leader might provide false proofs or fail to respond. If an input is not available while a maliciousleader still insists on signing its legality and providing proof-of-acceptance, other shards are likely to acceptthis proof. In this case, a double-spending attack is likely to be successful. The consistency and securityof the entire blockchain protocol will be severely damaged. • Transaction input being locked.
Letting a client act as a coordinator could lead to an input being lockedpermanently if the client fails to send the corresponding proof to related input shards. This happens whena client is malicious or offline. Note that in a blockchain system, for a single transaction, its inputs areusually owned by the same entity. However, in some cases such as crowdfunding transactions, multipleinputs of a transaction might belong to different individuals. In this way, if a client does not providethe corresponding availability certificate, it could cause the transaction input to be locked and affect theliveness property of the system. • Increased burden on the client.
If a client is responsible for collecting and forwarding availability cer-tificates, the client needs to record the shard state in the network and the IP addresses of participatingnodes to communicate with the corresponding committee leader. Hence, the storage and communicationoverhead of a client is increased a lot, which is not desired. Especially for lightweight clients, such asthose installed on mobile phones and smart homes, this overhead is impractical.Therefore, client-driven 2PC approaches might have certain problems and are not widely used.48 hard-driven 2PC.
In shard-driven 2PC based existing approaches, there are also problems to be solved. • Multiple calls of the BFT algorithm.
In most shard-driven 2PC approaches, to commit a single transac-tion, input and output shard committees need to run multiple times of BFT algorithm. For example, inChainspace [102], in the prepare phase, every input shard needs to run one BFT algorithm, and in thecommit phase, every input and output shard also have to invoke the BFT algorithm. Each BFT call re-quires communication among the members of the entire committee, which will cause more communicationand computation overhead to each node. So how to design a method to process multiple transactions ina single time is one of the future research directions. • Possible attacks.
Shard-driven 2PC approaches might suffer from different types of attacks. For example,the replay attack is proposed in [188] against cross-shard transactions. The essence of the attack is touse the previously generated transaction input availability certificate to disguise it as a proof of othertransaction input. The way to prevent this kind of attack is simple: attach the relevant transactionID information to the input availability certificate. Another type of attack is the transaction floodingattack where an adversary might manufacture a large number of transactions processed by a certainshard to cause the system liveness to be broken. Nguyen et al. [189] propose OptChain which utilizes anovel transaction assignment method to distribute transactions to different shards. The method could beapplied to common sharding blockchains to realize better performance. In summary, many cross-shardtransaction processing methods might be at risk of being attacked. In the future, it is necessary to analyzeother possible attacks and design more secure processing methods. • Malicious coordinators.
In 2PC based cross-shard transaction processing approaches, the role of thecoordinator is very important. In shard-driven 2PC based solutions, the coordinator is generally a leaderof the input or output shard. If a coordinator is malicious, it may delay the collection and forwardingprocess of the availability certificates, or even censor some of them. How to prevent the possible maliciousbehaviors of a coordinator is one of the directions of future research.
In transaction split based approach shown in Fig. 15, e.g., RapidChain [59], there might exist severalpossible problems.
Figure 15: Transaction split based approaches. • The way to generate a specific public key managed by 𝐶 𝑜𝑢𝑡 is unclear. The rule determining which shardis responsible for managing an input is not given. In the general case, it is based on public keys. In otherwords, the hash value of the last few digits of a public key address of a transaction input determines whichshard should manage the input. However, in the process of transaction split, it is necessary to generatetwo or more public key addresses that belong to 𝐶 𝑜𝑢𝑡 , that is, 𝐼 (cid:48) of tx and 𝐼 (cid:48) of tx in the above instance.49he method to generate such a public key address that satisfies the special requirement is not obvious,and the details of how to implement this process are not given. • The new generated outputs might be illegally spent. 𝐼 (cid:48) and 𝐼 (cid:48) are supposed to be generated by 𝐶 𝑜𝑢𝑡 , whilethe specific generation process is not given, that is, whether a leader of 𝐶 𝑜𝑢𝑡 is responsible for generating 𝐼 (cid:48) and 𝐼 (cid:48) , or whether it is generated by the entire committee 𝐶 𝑜𝑢𝑡 running BFT. As we know, 𝐼 (cid:48) and 𝐼 (cid:48) are public key addresses, and behind them, there are corresponding private keys. How to ensure theprivate keys are only known to the owners is not analyzed. If a committee member or the leader of 𝐶 𝑜𝑢𝑡 knows the private keys, 𝐼 (cid:48) and 𝐼 (cid:48) could be illegally spent. Also, if one of tx and tx is illegal, for example, tx successes and tx fails due to the unavailability of 𝐼 , then the way to retrieve the private key of 𝐼 (cid:48) forthe client is not given. • The method to handle multi-output transactions is not given.
In practical applications, some transactionsmay include multiple outputs. For example, the outputs of a transaction may include change returnedto the payer. In this case, the method for splitting a multi-input multi-output cross-shard transactioninto multiple single-input single-output transactions is not given, and the whole process will be verycomplicated. • The number of transactions is increased.
Transaction split leads to an increase in the number of transac-tions (at least three times as much as before), so the processing and storage overhead of the entire systemis largely increased.In summary, the specific implementation details described above are not given, yet are critical to thesystem security. In order for the transaction split based solutions to be more widely and securely applied,the implementation details need to be further studied and designed.
There might exist some problems in relay transaction based approaches. • Transaction confirmation delay is long.
For a cross-shard transaction, it first needs to get 𝜆 -confirmationby the input shard. Then the corresponding relay transaction is required to obtain 𝜆 -confirmation bythe output shard. After this, the cross-shard transaction is regarded as completed, then account 𝐵 couldtruly own the money and spend it. Therefore, the entire confirmation delay of a cross-shard transactionis equal to the time that at least 2 𝜆 blocks are committed. • Multi-input transactions could not be easily processed by relay transaction based approaches.
Monoxide[60] uses an account-based transaction model. Its cross-shard transaction processing method is actuallya simplified version, which can only handle single-input single-output cross-shard transactions. In fact,whether it is an account-based model or a UTXO-based model, there might be multi-input cross-shardtransactions. Taking the account-based model as an example, a user might have multiple different ac-counts, which are controlled by different shards. When the user initiates a transaction, he might usemultiple accounts as inputs to complete a single transaction. This cannot be achieved in Monoxide [60].To process multi-input cross-shard transactions in an eventual sharding blockchain, a two-phase commitmechanism should be introduced, since some inputs might not be available. In an eventual shardingblockchain, the confirmation of transactions and availability certificates is not instant, and it is difficult toemploy a lock-unlock mechanism for transaction inputs. In this way, an integral cross-shard transactionprocessing method will become extremely complicated in an eventual sharding blockchain, which is anopen research direction in the future. • Once a fork appears in a blockchain, the security of the system will be severely damaged.
Eventual shardingblockchains are different from instant sharding blockchains. In each shard, the block confirmation isprobabilistic. Even if a block gets 𝜆 -confirmation, the blockchain where the block is located still hasa certain probability of being replaced by another blockchain, that is, a fork occurs. In this case, thecross-shard transactions that are considered valid previously might become invalid, while the output of50he cross-shard transaction may have been spent by the owner. In this case, it is very difficult to roll backthe system, and the security of the system is seriously damaged.
9. Shard Reconfiguration
In this section, we introduce methods to conduct shard reconfiguration. In Section 9.1, the reason,purpose, and basic steps of shard reconfiguration are explained. Section 9.2 provides a taxonomy of existingapproaches for shard reconfiguration. All approaches are divided into reconfiguration through randomreplacement and under specific rules. Basic steps as well as typical approachesd for each category are given.Section 9.3 points out the possible security and efficiency problems with regard to shard reconfiguration.
We first explain the reason why a sharding blockchain needs to be reconfigured. Most blockchain systemsrequire regular reconfiguration, including the update of shard members and state transmission between oldand new shard members. This is because an adversary could launch a corruption attack by controlling acertain node after a certain period of time. If the members of a committee remain constant, the proportionof committee members controlled by an adversary may exceed a pre-defined safety threshold such as 1 / 𝑒 , the nodes thatintend to participate in epoch 𝑒 + 𝑒 , the replacement arrangement of the old and new committee members are determined, that is,which part of the old members in each shard is replaced by which new nodes. Third, the new members ofeach shard communicate with the corresponding old members and get the shard UTXO data and historicaltransaction data. Fourth, the old members stop working, the new members start to process transactions asnormal, and the whole protocol enters into epoch 𝑒 + We divide existing approaches into reconfiguration through random replacement and under specific rules.
Reconfiguration through random replacement means the selection of old members is a random procedure,i.e., each old member has an identical probability to be replaced. In general, reconfiguration through randomreplacement needs the following steps.First, in epoch 𝑒 , a parameter 𝑘 that represents the number of members to be replaced is determined bythe protocol. Then, after the node selection and allocation process, the new node list newnodes is confirmed.Third, for each shard, a seed 𝑠𝑒𝑒𝑑 𝑐 = 𝐻 ( 𝑐 || 𝜉 𝑒 ) is derived, where 𝑐 is the shard sequence number and 𝜉 𝑒 is arandomness. Then for every shard, its seed is used as one of the inputs of a pseudorandomness generator[84] function 𝑃𝑒𝑟𝑚 ( 𝑠𝑒𝑒𝑑 𝑐 , 𝑛 ) , to generate a permutation 𝜋 𝑐 . 𝜋 𝑐 could be used as an indicator to select 𝑘 nodes out of 𝑛 old shard members. In this way, the 𝑘 old members to be replaced in each shard are selected.The new nodes could be assigned to 𝑚 shards in a similar way. A seed 𝑠𝑒𝑒𝑑 = 𝐻 ( || 𝜉 𝑒 ) is first generated anda permutation 𝜋 could be obtained by computing 𝑃𝑒𝑟𝑚 ( 𝑠𝑒𝑒𝑑, 𝑚𝑘 ) since there are 𝑚𝑘 new nodes in the list51 ewnodes . At this time, the new committees for epoch 𝑒 + 𝑒 + 𝑘 is set as log 𝑛𝑚 . SGX-Sharding [190] also uses the same reconfiguration rule and parameter. The abovetwo schemes rely on the epoch randomness generated by their protocols as the random seed to complete therandom replacement process. The update procedure of committee members could rely on other specific rules. We summarize theserules into the following two categories.
Chronological rule.
This rule means that the node replacement is based on a chronological order, whichis similar to the sliding window rule. Specifically, a new node replaces an old node that has been inthe committee for the longest time. In some protocols such as Solida [57], every time a node is added, thecommittee performs a reconfiguration, and the newest node will replace the oldest one. Similarly, committeemembers could be divided into several parts according to the time when they join the committee. Each timea reconfiguration takes place, new nodes that find PoW solutions successfully replace the oldest 1 / / 𝑘 )of the existing committee members. This replacement rule is adopted in PaLa [103], Byzcoin [56], etc. Bounded cuckoo rule.
Reconfiguration under the bounded Cuckoo rule is shown in Fig. 16. This rule dynam-ically adjusts the members of each committee based on the number of active members in each committee.At the end of epoch 𝑒 , all committees are sorted according to the activeness level of all nodes, that is, thetotal number of transactions processed during the epoch. The 1 / 𝐴 (for “active”), and the remaining committees are placed into a set 𝐼 (for “inactive”).Then, the new nodes confirmed by the node selection mechanism are randomly assigned to a committee inthe set 𝐴 according to the epoch randomness 𝜉 𝑒 . After all the new nodes are allocated, 𝑘 nodes are selectedfrom each committee of set 𝐴 , and these nodes will be kicked out of these committees and randomly assignedto committees in the set 𝐼 . The reconfiguration procedure is done at this moment, and newly confirmedcommittees start to process transactions normally. RapidChain [59] utilizes the bounded Cuckoo rule tocomplete committee reconfiguration. Figure 16: Reconfiguration under bounded Cuckoo rule. .3. Problems and Future Directions We summarize potential problems that might occur in the existing approaches during the shard recon-figuration process. 𝜏𝜏 is used to denote the time to complete a corruption process for an adversary, which is a crucialparameter in shard reconfiguration determining the system security. If an adversary could complete itscorruption attack before the committees complete their reconfiguration, then the adversary might controlthe committee and destroy the system security. Generally speaking, the following condition applies forsharding blockchains that use PoW to select shard members: 𝜏 > 𝑇 𝑒𝑝𝑜𝑐ℎ where 𝑇 𝑒𝑝𝑜𝑐ℎ denotes the time of an epoch. In sharding blockchains that use PoW to select members, nodesthat want to participate in the protocol mine in some epoch 𝑒 . Then nodes finding PoW solutions shouldsubmit their results to the reference committee or broadcast the results. So in the mining process in epoch 𝑒 , the identities of new nodes are exposed to an adversary who can launch a corruption attack from this timeon. When the protocol enters into epoch 𝑒 +
1, new nodes become committee members and work normally.So it is required that an adversary could not complete its corruption attack until epoch 𝑒 + 𝜏 > 𝑇 𝑒𝑝𝑜𝑐ℎ . 𝑇 𝑒𝑝𝑜𝑐ℎ is related to the speed tofind a certain number of PoW solutions and may vary in different sharding blockchains. In existing shardingblockchain schemes, there is no discussion about the corruption time parameter. The analysis of it is one ofthe future research directions. A new shard member should download historical data of the corresponding shard [191]. This procedure isusually called bootstrapping in some literature. During the bootstrapping, there are two potential problemsfor new shard members. One is the security problem, where an adversary might produce some fake data toconvince a new member. This kind of attack is easy to launch in a PoS-based blockchain since the cost offorging a fake chain is relatively low. The other problem is about the performance, that is, the downloadingof a large amount of data takes a long time, making the system not work properly during reconfiguration.So how to ensure that honest committee members could confirm the correctness of the data received, andimprove the efficiency of the entire reconfiguration process is worthy of in-depth study in the future.
In reconfiguration solutions through random replacement, only a fraction of old members is replacedby new nodes. However, the analysis of random assignment in existing schemes [58] assumes replacing allcommittee members instead of a certain proportion of members, so the analysis results are not accurate.Every time some members are replaced, it should be guaranteed that the honest members occupy more thana certain proportion of the entire committee. The security analysis of the reconfiguration process needs tobe more rigorous, taking all shards into account.In reconfiguration solutions under the chronological rule, an adversary could launch targeted corruptionattacks on relatively new nodes. In this way, an adversary with an identical corruption attack ability couldhave a higher probability of controlling the committee. In other words, a stricter requirement is in need ofan adversary’s corruption parameter, and the security level of the system will become lower.As for the bounded Cuckoo rule, the committee members in set 𝐼 are constantly increasing, and thereis no explanation of how to kick out old members in 𝐼 . The increase of committee members will lead toexcessive system overhead of the BFT algorithm. Furthermore, the scenario is not practical in generalsharding blockchains. The reason is that, in order to prevent the transaction flood attack, the transactionsin a sharding blockchain are randomly allocated to different shards. The transaction flood attack meansthat an adversary creates a large number of transactions supposed to be managed by a particular shard.Therefore, the actual number of transactions processed by each shard is similar, and there is no obvious gapto distinguish them. 53 .3.4. Initial Setup of the Protocol In the initial phase of a sharding blockchain protocol, the method to safely initialize and create multiplegenesis blocks should be designed. This process is usually called the protocol setup. The protocol setupproblem also exists in other general blockchains. Most existing blockchain protocols assume a trusted setup,where information such as the genesis block is set by a trusted third party. In sharding blockchains, theprotocol setup is different from that of general blockchains since it is necessary to set the genesis block andgenesis committee members for each shard. The details of these settings deserve a more in-depth study.Besides, the way to initialize the protocol without relying on trusted setup [192, 193] such as using securemulti-party computing should be studied in deep.
10. Motivation Mechanism
In this section, we introduce motivation mechanisms that are important in sharding blockchain systems.Section 10.1 explains the meaning and purpose of a motivation mechanism. Section 10.2 summarizes therelated research from the following aspects, rewards for block producers and leader rewards, penalties fornegative behaviors, and rewards based on reputation. Finally, we analyze the related problems and futureresearch directions in Section 10.3.
Blockchain systems need to design a motivation mechanism to encourage nodes to participate in theprotocol. In general, the more nodes in the network, the safer the system. Besides, the node participatingin the protocol needs to consume a certain amount of communication bandwidth and computational power.If the corresponding reward is not obtained, the node will lose the motivation to participate in the protocol.The difficulty in designing a motivation mechanism is to ensure that the rewards received by each nodeare fair and that malicious behaviors can be punished accordingly. In addition, when analyzing motivationmechanisms, it is usually necessary to assume that all nodes are rational, i.e., each node’s behavior is tomaximize its interests.Motivation mechanisms consist of incentive and penalty parts. Incentive mechanisms refer to the rewardsfor certain contributions of participating nodes in the blockchain. The rewards are generally in the form oftokens in the system. At the same time, certain nodes need to be punished by a penalty mechanism fornegative sabotage or malicious behaviors. Generally speaking, in public blockchains, motivation mechanismsneed to be carefully designed to encourage nodes to participate in the operations of the protocol.Since the motivation mechanism is not the focus of this paper, and there are very few studies on theincentive mechanism of sharding blockchains, we only briefly introduce the motivation mechanism.
We classify motivation mechanisms into rewards for blockchain producers and leader, penalties for neg-ative behaviors, and rewards based on reputation.
There are multiple shards in a sharding blockchain, and each shard maintains its own blockchain. Eachshard is constantly producing blocks, and the block producers should receive corresponding rewards. Theblock producer here might vary depending on the blockchain protocol. In an eventual sharding blockchain,such as Monoxide [60] and Parallel Chains [89], a block producer is an individual node. In this case, asingle node will get a complete block reward. In instant sharding blockchains, the committee in each shardruns an intra-shard consensus algorithm to generate blocks, so all members in the committee should get thecorresponding block rewards. The rewards could be further distributed fairly according to the contributionsof each member, i.e., the amount of communication and the number of transactions processed.In instant sharding blockchains, a committee in a shard runs the BFT algorithm, which requires thecollaboration of a leader. In addition, a leader is usually responsible for collecting, merging, and forwardingof messages within the committee, so its computation and communication costs are higher than the other54ommittee members. Therefore, the incentive mechanism within the committee should be more elaboratelydesigned to ensure the fairness of reward distribution.Wang and Wu [106] propose Lever as an incentive mechanism to distribute rewards among rationalstakeholders in BFT consensus algorithms, and they further apply it to the sharding blockchain. Manshaei et al. [194] conduct an analysis of the motivation mechanisms in sharding blockchains based on the gametheory. They propose an incentive-compatible reward mechanism to encourage shard members to participatein the protocol.
In a sharding blockchain, some negative behaviors need to be punished. Negative behaviors could befurther divided into two categories. One is sabotage behaviors, which can also be understood as passivebehaviors, such as committee members not voting on the leader’s valid proposals, or a leader not proposingnew blocks or transactions within a specified time. The other is malicious behaviors, such as a leaderproposing two different proposals (blocks or transactions) in the same round, or the block proposed by aleader contains invalid transactions. Besides, committee members might vote for more than one proposalmessage in the same round. In general, the establishment of a penalty mechanism first requires nodes tosubmit a certain deposit before participating in the protocol, e.g., Casper FFG [52]. When a maliciousbehavior is detected, the deposit may be deducted accordingly.
The reputation based motivation mechanism originates from the cooperative P2P scenarios, e.g., dis-tributed storage systems [195] and is introduced to the blockchain area [196]. Reputation usually refers to theunified evaluation and scoring of all nodes based on the past performances of participating nodes, includingcorresponding time, the number of transactions processed, and the number of malicious behaviors that arerelated to the nodes. In this process, different behaviors may come with different evaluation weights. Eachparticipating node may eventually get a score, and the score will be updated as the system advances. Whennodes perform well and participate actively, their rating scores will grow. On the contrary, if a node losesresponse or is detected to behave maliciously, then the node’s score will decrease. The rewards in the systemwill be distributed according to each node’s score. Nodes with higher reputation will get more rewards.Bugday et al. [197] apply learning methods to the establishment of node reputation in the blockchain,where the node reputation is dynamically changing. Huang et al. [104] utilize high incentives to motivatenodes to behave themselves in the sharding blockchain. Similarly, the authors adopt a reputation-basedevaluation system to measure the different processing capabilities of each node. CycLedger [105] mainlyuses the reputation to evaluate the mining ability of each miner. Block rewards are allocated according tothe reputation value of each miner.
The establishment of a motivation mechanism needs to take the differences between sharding blockchainsand ordinary blockchains into account.First, according to the various intra-committee consensus algorithms, the distribution of rewards forcommittee members should be more specifically designed to realize fairness. In algorithms, especially thosethat adopt a stable leader such as PBFT [43], The leader is of vital importance to the stable operation ofthe system. In each round, the leader is responsible for broadcasting a proposal and collecting votes in thesystem. Consequently, the leader will have a higher communication and computation overhead than othermembers, so it should obtain higher rewards. However, in this way, all members will hope to become aleader, and malicious nodes might take the opportunity to launch a view-change operation to replace theold leader. Moreover, an adversary might launch a DoS attack or network partition attack against theleader, causing its network to be paralyzed and replaced by a new leader. Relatively speaking, in the BFTalgorithms that employ a continuously changing leader such as HotStuff [101], it is easier for the incentivemechanism to achieve fairness. 55econd, the influences caused by cross-shard transaction processing need to be considered. A coordinatoris responsible for the forwarding of availability certificates, that is, to accomplish the communication amongshards. The work of a coordinator needs to be carefully considered when designing a motivation mechanism.There may also be some malicious behaviors. For example, in a client-driven 2PC method, a client mightnot forward the transaction input availability certificate within a specified time. In sharding blockchainsusing shard-driven 2PC, an input shard leader might not forward availability certificates to other relatedshards.
As far as we know, no detailed theoretical analysis has been conducted on the motivation mechanismof sharding blockchains. The analysis requires a certain financial foundation, e.g., Nash equilibrium the-ory [198]. Therefore, for the sharding blockchains, a reasonable, comprehensive, and secure motivationmechanism and strict analysis of it are the future research directions.
11. Related Work
In the following, we introduce the related work from different perspectives.
Wang et al. [65] give an overview of the research on sharding blockchain systems. However, their classifi-cation of the sharding blockchain components is abstract, and they do not provide a more in-depth analysisfor each component. In contrast, in our paper, we provide a more complete characterization of componentsand their composition into a sharding blockchain system. Moreover, for each independent component, weprovide a taxonomy of the existing approaches for that component. In addition, we deeply analyze possibleproblems and future research directions, which are of great importance to related researchers.Yu et al. [199] take each existing scheme as a starting point and give basic details of each shardingblockchain scheme. However, they do not provide a macro vision and overall classification of shardingblockchain systems. Our paper provides a systematic taxonomy, and we also analyze related research onthe security model and motivation mechanism of sharding blockchain systems that are not mentioned in theprevious research.
Avarikioti et al. [108] provide a relatively formalized analysis of sharding blockchain systems, and proposesome evaluation indicators, including communication, computation, storage complexity, and the scalingfactor. Zamyatin et al. [64] analyze communication across distributed ledgers. Their analysis includes cross-shard communication between homogeneous blockchains and cross-chain swap such as sidechains betweenheterogeneous blockchains. Han et al. [200] analyze shard allocation protocols for sharding blockchainsystems. These studies mainly focus on a certain part of sharding blockchains, while our study combines aunified framework with thorough component analysis.
Bano et al. [47, 201] study consensus mechanisms in the age of blockchain, including classical consensus,proof-of-X consensus, and hybrid consensus. Garay and Kiayias [202] provide a consensus taxonomy inthe blockchain era based on different network, setup, and computational assumptions. In their paper,consensus mechanisms are divided into consensus in the point-to-point setting, consensus in the peer-to-peersetting, and ledger consensus. Alsunaidi and Alhaidari [203] conduct a comprehensive survey on popularblockchain consensus algorithms, focusing on their security and performance. Nguyen and Kim [204] classifyexisting blockchain consensus algorithms into proof-based consensus and voting-based consensus. Xiao etal. [205] carry out a survey on blockchain consensus protocols and identify five core components, namelyblock proposal, block validation, information propagation, block finalization, and incentive mechanism. Bycontrast, our study conceptualizes functional components of sharding blockchain systems, going beyondconsensus. 56
Some studies formalize specific properties to be followed by different solutions to the scalability of block-chain systems, but without aiming at a broad review as our study does. Xie et al. [206] study the scalabilityproblem of blockchain from the perspectives of throughput, storage, and networking. Kim et al. [207] an-alyze existing solutions to realize blockchain scale-out and classify them into different types, i.e., on-chain,off-chain, side-chain, child-chain, and inter-chain. Zhou et al. [208] summarize existing scaling schemes andprovide potential research directions for solving the blockchain scalability problem.
12. Conclusion
This paper decomposes sharding blockchains into multiple components and analyzes the basic concepts,existing approaches, and potential problems of each component. On this basis, designing a new shardingblockchain system could be simplified into composing several different components. In this way, each com-ponent could be improved separately according to the current latest research, and the improved componentcould be integrated into a whole sharding blockchain system without affecting the security of other partsand the entire system. For each component, the possible problems and future research directions that areproposed in our paper are worthy of attention. We believe that our systematic and comprehensive researchon sharding blockchains could give insights to future researchers.
Acknowledgment
Our deepest gratitude goes to the editors and reviewers for their careful work and meaningful suggestionsthat help improve this paper. This work is supported in part by the National Key R&D Program of China(2017YFB1400702), in part by the National Natural Science Foundation of China (61972017, 61972018,61972014, 72031001), in part by the National Cryptography Development Fund (MMJJ20180215), and inpart by the Fundamental Research Funds for the Central Universities (YWF-20-BJ-J-1039).
References [1] S. Nakamoto, Bitcoin: A peer-to-peer electronic cash system, https://bitcoin.org/bitcoin.pdf (2008).[2] D. Johnson, A. Menezes, S. A. Vanstone, The elliptic curve digital signature algorithm (ECDSA), Int. J. Inf. Sec. 1 (1)(2001) 36–63.[3] D. Boneh, M. Drijvers, G. Neven, Compact multi-signatures for smaller blockchains, in: Advances in Cryptology -ASIACRYPT 2018 - 24th International Conference on the Theory and Application of Cryptology and InformationSecurity, Brisbane, QLD, Australia, December 2-6, 2018, Proceedings, Part II, 2018, pp. 435–464.[4] J. Coron, Y. Dodis, C. Malinaud, P. Puniya, Merkle-damg˚ard revisited: How to construct a hash function, in: Advancesin Cryptology - CRYPTO 2005: 25th Annual International Cryptology Conference, Santa Barbara, California, USA,August 14-18, 2005, Proceedings, 2005, pp. 430–448.[5] J. Bonneau, A. Miller, J. Clark, A. Narayanan, J. A. Kroll, E. W. Felten, Sok: Research perspectives and challenges forbitcoin and cryptocurrencies, in: 2015 IEEE Symposium on Security and Privacy, SP 2015, San Jose, CA, USA, May17-21, 2015, 2015, pp. 104–121.[6] M. S. Ali, M. Vecchio, M. Pincheira, K. Dolui, F. Antonelli, M. H. Rehmani, Applications of blockchains in the internetof things: A comprehensive survey, IEEE Communications Surveys and Tutorials 21 (2) (2019) 1676–1717.[7] M. A. Khan, K. Salah, Iot security: Review, blockchain solutions, and open challenges, Future Gener. Comput. Syst. 82(2018) 395–411.[8] K. Gai, J. Guo, L. Zhu, S. Yu, Blockchain meets cloud computing: A survey, IEEE Communications Surveys and Tutorials22 (3) (2020) 2009–2030.[9] R. Yang, F. R. Yu, P. Si, Z. Yang, Y. Zhang, Integrated blockchain and edge computing systems: A survey, some researchissues and challenges, IEEE Communications Surveys and Tutorials 21 (2) (2019) 1508–1532.[10] J. Xie, H. Tang, T. Huang, F. R. Yu, R. Xie, J. Liu, Y. Liu, A survey of blockchain technology applied to smart cities:Research issues and challenges, IEEE Communications Surveys and Tutorials 21 (3) (2019) 2794–2830.[11] B. Egelund-M¨uller, M. Elsman, F. Henglein, O. Ross, Automated execution of financial contracts on blockchains, Business& Information Systems Engineering 59 (6) (2017) 457–467, nominated for Association of Information Systems Best PaperAward 2017.[12] P. C. Treleaven, R. G. Brown, D. Yang, Blockchain technology in finance, Computer 50 (9) (2017) 14–17.[13] I. Eyal, Blockchain technology: Transforming libertarian cryptocurrency dreams to finance and banking realities, Com-puter 50 (9) (2017) 38–49.
14] K. Korpela, J. Hallikas, T. Dahlberg, Digital supply chain transformation toward blockchain integration, in: 50th HawaiiInternational Conference on System Sciences, HICSS 2017, Hilton Waikoloa Village, Hawaii, USA, January 4-7, 2017,2017, pp. 1–10.[15] T. Neudecker, H. Hartenstein, Network layer aspects of permissionless blockchains, IEEE Communications Surveys andTutorials 21 (1) (2019) 838–857.[16] M. Saad, J. Spaulding, L. Njilla, C. A. Kamhoua, S. Shetty, D. Nyang, A. Mohaisen, Exploring the attack surface ofblockchain: A comprehensive survey, IEEE Communications Surveys and Tutorials 22 (3) (2020) 1977–2008.[17] M. Conti, S. K. E, C. Lal, S. Ruj, A survey on security and privacy issues of bitcoin, IEEE Communications Surveys andTutorials 20 (4) (2018) 3416–3452.[18] J. A. Garay, A. Kiayias, N. Leonardos, The bitcoin backbone protocol: Analysis and applications, in: Advances in Cryp-tology - EUROCRYPT 2015 - 34th Annual International Conference on the Theory and Applications of CryptographicTechniques, Sofia, Bulgaria, April 26-30, 2015, Proceedings, Part II, 2015, pp. 281–310.[19] R. Pass, L. Seeman, A. Shelat, Analysis of the blockchain protocol in asynchronous networks, in: Advances in Cryptology -EUROCRYPT 2017 - 36th Annual International Conference on the Theory and Applications of Cryptographic Techniques,Paris, France, April 30 - May 4, 2017, Proceedings, Part II, 2017, pp. 643–673.[20] A. Hafid, A. S. Hafid, M. Samih, Scaling blockchains: A comprehensive survey, IEEE Access 8 (2020) 125244–125262.[21] J. Poon, T. Dryja, The bitcoin lightning network: Scalable off-chain instant payments, (2016).[22] A. Kiayias, O. S. T. Litos, A composable security treatment of the lightning network, in: 33rd IEEE Computer SecurityFoundations Symposium, CSF 2020, Boston, MA, USA, June 22-26, 2020, 2020, pp. 334–349.[23] G. Malavolta, P. Moreno-Sanchez, A. Kate, M. Maffei, S. Ravi, Concurrency and privacy with payment-channel networks,in: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS 2017, Dallas,TX, USA, October 30 - November 03, 2017, 2017, pp. 455–471.[24] R. Khalil, A. Gervais, Revive: Rebalancing off-blockchain payment networks, in: Proceedings of the 2017 ACM SIGSACConference on Computer and Communications Security, CCS 2017, Dallas, TX, USA, October 30 - November 03, 2017,2017, pp. 439–453.[25] S. Dziembowski, L. Eckey, S. Faust, D. Malinowski, Perun: Virtual payment hubs over cryptocurrencies, in: 2019 IEEESymposium on Security and Privacy, SP 2019, San Francisco, CA, USA, May 19-23, 2019, 2019, pp. 106–123.[26] P. Gazi, A. Kiayias, D. Zindros, Proof-of-stake sidechains, in: 2019 IEEE Symposium on Security and Privacy, SP 2019,San Francisco, CA, USA, May 19-23, 2019, 2019, pp. 139–156.[27] A. Kiayias, D. Zindros, Proof-of-work sidechains, in: Financial Cryptography and Data Security - FC 2019 InternationalWorkshops, VOTING and WTSC, St. Kitts, St. Kitts and Nevis, February 18-22, 2019, Revised Selected Papers, 2019,pp. 21–34.[28] J. Poon, V. Buterin, Plasma: Scalable autonomous smart contracts, (2017).[29] Y. Liu, J. Liu, Z. Zhang, T. Xu, H. Yu, Overview on consensus mechanism of blockchain technology, Journal of CryptologicResearch 6 (4) (2019) 395–432.[30] J. C. Corbett, J. Dean, M. Epstein, A. Fikes, C. Frost, J. J. Furman, S. Ghemawat, A. Gubarev, C. Heiser, P. Hochschild,W. C. Hsieh, S. Kanthak, E. Kogan, H. Li, A. Lloyd, S. Melnik, D. Mwaura, D. Nagle, S. Quinlan, R. Rao, L. Rolig,Y. Saito, M. Szymaniak, C. Taylor, R. Wang, D. Woodford, Spanner: Google’s globally distributed database, ACMTrans. Comput. Syst. 31 (3) (2013) 8:1–8:22.[31] L. Luu, V. Narayanan, C. Zheng, K. Baweja, S. Gilbert, P. Saxena, A secure sharding protocol for open blockchains,in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria,October 24-28, 2016, 2016, pp. 17–30.[32] D. Deuber, B. Magri, S. A. K. Thyagarajan, Redactable blockchain in the permissionless setting, in: 2019 IEEE Sympo-sium on Security and Privacy, SP 2019, San Francisco, CA, USA, May 19-23, 2019, 2019, pp. 124–138.[33] G. Ateniese, B. Magri, D. Venturi, E. R. Andrade, Redactable blockchain - or - rewriting history in bitcoin and friends,in: 2017 IEEE European Symposium on Security and Privacy, EuroS&P 2017, Paris, France, April 26-28, 2017, 2017,pp. 111–126.[34] F. Reid, M. Harrigan, An analysis of anonymity in the bitcoin system, in: PASSAT/SocialCom 2011, Privacy, Security,Risk and Trust (PASSAT), 2011 IEEE Third International Conference on and 2011 IEEE Third International Conferenceon Social Computing (SocialCom), Boston, MA, USA, 9-11 Oct., 2011, 2011, pp. 1318–1326.[35] M. C. K. Khalilov, A. Levi, A survey on anonymity and privacy in bitcoin-like digital cash systems, IEEE CommunicationsSurveys and Tutorials 20 (3) (2018) 2543–2585.[36] D. Chaum, Blind signature system, in: Advances in Cryptology, Proceedings of CRYPTO ’83, Santa Barbara, California,USA, August 21-24, 1983, 1983, p. 153.[37] E. Heilman, F. Baldimtsi, S. Goldberg, Blindly signed contracts: Anonymous on-blockchain and off-blockchain bitcointransactions, in: Financial Cryptography and Data Security - FC 2016 International Workshops, BITCOIN, VOTING,and WAHC, Christ Church, Barbados, February 26, 2016, Revised Selected Papers, 2016, pp. 43–60.[38] F. Zhang, K. Kim, Id-based blind signature and ring signature from pairings, in: Advances in Cryptology - ASIACRYPT2002, 8th International Conference on the Theory and Application of Cryptology and Information Security, Queenstown,New Zealand, December 1-5, 2002, Proceedings, 2002, pp. 533–547.[39] S. Sun, M. H. Au, J. K. Liu, T. H. Yuen, Ringct 2.0: A compact accumulator-based (linkable ring signature) protocol forblockchain cryptocurrency monero, in: Computer Security - ESORICS 2017 - 22nd European Symposium on Researchin Computer Security, Oslo, Norway, September 11-15, 2017, Proceedings, Part II, 2017, pp. 456–474.
40] S. Bowe, A. Gabizon, M. D. Green, A multi-party protocol for constructing the public parameters of the pinocchiozk-snark, in: Financial Cryptography and Data Security - FC 2018 International Workshops, BITCOIN, VOTING, andWTSC, Nieuwpoort, Cura¸cao, March 2, 2018, Revised Selected Papers, 2018, pp. 64–77.[41] I. Miers, C. Garman, M. Green, A. D. Rubin, Zerocoin: Anonymous distributed e-cash from bitcoin, in: 2013 IEEESymposium on Security and Privacy, SP 2013, Berkeley, CA, USA, May 19-22, 2013, 2013, pp. 397–411.[42] R. Schollmeier, A definition of peer-to-peer networking for the classification of peer-to-peer architectures and applications,in: 1st International Conference on Peer-to-Peer Computing (P2P 2001), 27-29 August 2001, Link¨oping, Sweden, 2001,pp. 101–102.[43] M. Castro, B. Liskov, Practical byzantine fault tolerance, in: Proceedings of the Third USENIX Symposium on OperatingSystems Design and Implementation (OSDI), New Orleans, Louisiana, USA, February 22-25, 1999, 1999, pp. 173–186.[44] R. Beck, M. Avital, M. Rossi, J. B. Thatcher, Blockchain technology in business and information systems research, Bus.Inf. Syst. Eng. 59 (6) (2017) 381–384.[45] Y. Hei, Y. Liu, D. Li, J. Liu, Q. Wu, Themis: An accountable blockchain-based p2p cloud storage scheme, Peer-to-PeerNetworking and Applications (2020) 1–15.[46] S. Raval, Decentralized applications: harnessing Bitcoin’s blockchain technology, ” O’Reilly Media, Inc.”, 2016.[47] S. Bano, A. Sonnino, M. Al-Bassam, S. Azouvi, P. McCorry, S. Meiklejohn, G. Danezis, Sok: Consensus in the ageof blockchains, in: Proceedings of the 1st ACM Conference on Advances in Financial Technologies, AFT 2019, Zurich,Switzerland, October 21-23, 2019, 2019, pp. 183–198.[48] I. Eyal, A. E. Gencer, E. G. Sirer, R. van Renesse, Bitcoin-ng: A scalable blockchain protocol, in: 13th USENIXSymposium on Networked Systems Design and Implementation, NSDI 2016, Santa Clara, CA, USA, March 16-18, 2016,2016, pp. 45–59.[49] R. Pass, E. Shi, Fruitchains: A fair blockchain, in: Proceedings of the ACM Symposium on Principles of DistributedComputing, PODC 2017, Washington, DC, USA, July 25-27, 2017, 2017, pp. 315–324.[50] Y. Sompolinsky, A. Zohar, Secure high-rate transaction processing in bitcoin, in: Financial Cryptography and DataSecurity - 19th International Conference, FC 2015, San Juan, Puerto Rico, January 26-30, 2015, Revised SelectedPapers, 2015, pp. 507–527.[51] Y. Sompolinsky, Y. Lewenberg, A. Zohar, SPECTRE: A fast and scalable cryptocurrency protocol, http://eprint.iacr.org/2016/1159 (2016).[52] V. Buterin, V. Griffith, Casper the friendly finality gadget, http://arxiv.org/abs/1710.09437 (2017).[53] P. Daian, R. Pass, E. Shi, Snow white: Robustly reconfigurable consensus and applications to provably secure proof ofstake, in: Financial Cryptography and Data Security - 23rd International Conference, FC 2019, Frigate Bay, St. Kittsand Nevis, February 18-22, 2019, Revised Selected Papers, 2019, pp. 23–41.[54] A. Kiayias, A. Russell, B. David, R. Oliynykov, Ouroboros: A provably secure proof-of-stake blockchain protocol, in:Advances in Cryptology - CRYPTO 2017 - 37th Annual International Cryptology Conference, Santa Barbara, CA, USA,August 20-24, 2017, Proceedings, Part I, 2017, pp. 357–388.[55] C. Decker, J. Seidel, R. Wattenhofer, Bitcoin meets strong consistency, in: Proceedings of the 17th International Con-ference on Distributed Computing and Networking, Singapore, January 4-7, 2016, 2016, pp. 13:1–13:10.[56] E. Kokoris-Kogias, P. Jovanovic, N. Gailly, I. Khoffi, L. Gasser, B. Ford, Enhancing bitcoin security and performancewith strong consistency via collective signing, in: 25th USENIX Security Symposium, USENIX Security 16, Austin, TX,USA, August 10-12, 2016., 2016, pp. 279–296.[57] I. Abraham, D. Malkhi, K. Nayak, L. Ren, A. Spiegelman, Solida: A blockchain protocol based on reconfigurablebyzantine consensus, in: 21st International Conference on Principles of Distributed Systems, OPODIS 2017, Lisbon,Portugal, December 18-20, 2017, 2017, pp. 25:1–25:19.[58] E. Kokoris-Kogias, P. Jovanovic, L. Gasser, N. Gailly, E. Syta, B. Ford, Omniledger: A secure, scale-out, decentralizedledger via sharding, in: 2018 IEEE Symposium on Security and Privacy, SP 2018, Proceedings, 21-23 May 2018, SanFrancisco, California, USA, 2018, pp. 583–598.[59] M. Zamani, M. Movahedi, M. Raykova, Rapidchain: Scaling blockchain via full sharding, in: Proceedings of the 2018ACM SIGSAC Conference on Computer and Communications Security, CCS 2018, Toronto, ON, Canada, October 15-19,2018, 2018, pp. 931–948.[60] J. Wang, H. Wang, Monoxide: Scale out blockchains with asynchronous consensus zones, in: 16th USENIX Symposiumon Networked Systems Design and Implementation, NSDI 2019, Boston, MA, February 26-28, 2019, 2019, pp. 95–112.[61] H. Yu, I. Nikolic, R. Hou, P. Saxena, OHIE: blockchain scaling made simple, in: 2020 IEEE Symposium on Security andPrivacy, SP 2020, San Francisco, CA, USA, May 18-21, 2020, 2020, pp. 90–105.[62] V. K. Bagaria, S. Kannan, D. Tse, G. C. Fanti, P. Viswanath, Prism: Deconstructing the blockchain to approach physicallimits, in: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 2019,London, UK, November 11-15, 2019, 2019, pp. 585–602.[63] A. Tomescu, S. Devadas, Catena: Efficient non-equivocation via bitcoin, in: 2017 IEEE Symposium on Security andPrivacy, SP 2017, San Jose, CA, USA, May 22-26, 2017, 2017, pp. 393–409.[64] A. Zamyatin, M. Al-Bassam, D. Zindros, E. Kokoris-Kogias, P. Moreno-Sanchez, A. Kiayias, W. J. Knottenbelt, Sok:Communication across distributed ledgers, https://eprint.iacr.org/2019/1128 (2019).[65] G. Wang, Z. J. Shi, M. Nixon, S. Han, Sok: Sharding on blockchain, in: Proceedings of the 1st ACM Conference onAdvances in Financial Technologies, AFT 2019, Zurich, Switzerland, October 21-23, 2019, 2019, pp. 41–61.[66] C. Dwork, N. A. Lynch, L. J. Stockmeyer, Consensus in the presence of partial synchrony, J. ACM 35 (2) (1988) 288–323.[67] D. Dolev, C. Dwork, L. J. Stockmeyer, On the minimal synchronism needed for distributed consensus, J. ACM 34 (1)(1987) 77–97.
68] M. J. Fischer, N. A. Lynch, M. Paterson, Impossibility of distributed consensus with one faulty process, J. ACM 32 (2)(1985) 374–382.[69] H. Sukhwani, J. M. Mart´ınez, X. Chang, K. S. Trivedi, A. Rindos, Performance modeling of PBFT consensus processfor permissioned blockchain network (hyperledger fabric), in: 36th IEEE Symposium on Reliable Distributed Systems,SRDS 2017, Hong Kong, Hong Kong, September 26-29, 2017, 2017, pp. 253–255.[70] R. Cramer, I. Damg˚ard, S. Dziembowski, M. Hirt, T. Rabin, Efficient multiparty computations secure against an adaptiveadversary, in: Advances in Cryptology - EUROCRYPT ’99, International Conference on the Theory and Application ofCryptographic Techniques, Prague, Czech Republic, May 2-6, 1999, Proceeding, 1999, pp. 311–326.[71] R. Pass, E. Shi, Hybrid consensus: Efficient consensus in the permissionless model, in: 31st International Symposium onDistributed Computing, DISC 2017, October 16-20, 2017, Vienna, Austria, 2017, pp. 39:1–39:16.[72] Y. Gilad, R. Hemo, S. Micali, G. Vlachos, N. Zeldovich, Algorand: Scaling byzantine agreements for cryptocurrencies,in: Proceedings of the 26th Symposium on Operating Systems Principles, Shanghai, China, October 28-31, 2017, 2017,pp. 51–68.[73] B. Hu, Z. Zhang, J. Liu, Y. Liu, J. Yin, R. Lu, X. Lin, A comprehensive survey on smart contract construction andexecution: Paradigms, tools and systems, https://arxiv.org/abs/2008.13413 (2020).[74] S. King, S. Nadal, PPcoin: Peer-to-peer crypto-currency with proof-of-stake, https://cdn.bitturk.com/whitepaper/ppc.pdf .[75] E. Kokoris-Kogias, Robust and scalable consensus for sharded distributed ledgers, https://eprint.iacr.org/2019/676 (2019).[76] R. Pass, E. Shi, Hybrid consensus: Efficient consensus in the permissionless model, in: 31st International Symposium onDistributed Computing, DISC 2017, October 16-20, 2017, Vienna, Austria, 2017, pp. 39:1–39:16.[77] F. B. Schneider, Implementing fault-tolerant services using the state machine approach: A tutorial, ACM Comput. Surv.22 (4) (1990) 299–319.[78] F. Tschorsch, B. Scheuermann, Bitcoin and beyond: A technical survey on decentralized digital currencies, IEEE Com-munications Surveys and Tutorials 18 (3) (2016) 2084–2123.[79] M. J. Fischer, The consensus problem in unreliable distributed systems (A brief survey), in: Fundamentals of ComputationTheory, Proceedings of the 1983 International FCT-Conference, Borgholm, Sweden, August 21-27, 1983, 1983, pp. 127–140.[80] I. Abraham, K. Nayak, L. Ren, N. Shrestha, On the optimality of optimistic responsiveness, https://eprint.iacr.org/2020/458 (2020).[81] A. Momose, J. P. Cruz, Y. Kaji, Hybrid-bft: Optimistically responsive synchronous consensus with optimal latency orresilience, https://eprint.iacr.org/2020/406 (2020).[82] M. E. Fayad, D. C. Schmidt, R. E. Johnson, Building application frameworks: object-oriented foundations of frameworkdesign, John Wiley & Sons, Inc., 1999.[83] J. R. Douceur, The sybil attack, in: Peer-to-Peer Systems, First International Workshop, IPTPS 2002, Cambridge, MA,USA, March 7-8, 2002, Revised Papers, 2002, pp. 251–260.[84] M. Luby, Pseudorandomness and cryptographic applications, Princeton computer science notes, Princeton UniversityPress, 1996.[85] E. Androulaki, C. Cachin, A. D. Caro, E. Kokoris-Kogias, Channels: Horizontal scaling and confidentiality on permis-sioned blockchains, in: Computer Security - 23rd European Symposium on Research in Computer Security, ESORICS2018, Barcelona, Spain, September 3-7, 2018, Proceedings, Part I, 2018, pp. 111–131.[86] M. J. Amiri, D. Agrawal, A. E. Abbadi, Sharper: Sharding permissioned blockchains over network clusters, http://arxiv.org/abs/1910.00765 (2019).[87] M. J. Amiri, D. Agrawal, A. E. Abbadi, On sharding permissioned blockchains, in: IEEE International Conference onBlockchain, Blockchain 2019, Atlanta, GA, USA, July 14-17, 2019, 2019, pp. 282–285.[88] D. R. Lee, Y. Jang, H. Kim, Poster: A proof-of-stake (pos) blockchain protocol using fair and dynamic sharding man-agement, in: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 2019,London, UK, November 11-15, 2019, 2019, pp. 2553–2555.[89] M. Fitzi, P. Gazi, A. Kiayias, A. Russell, Proof-of-stake blockchain protocols with near-optimal throughput, https://eprint.iacr.org/2020/037 (2020).[90] T. Hanke, M. Movahedi, D. Williams, Dfinity technology overview series, consensus system, https://arxiv.org/pdf/1805.04548.pdf (2018).[91] Z. Team, et al., The zilliqa technical whitepaper, https://docs.zilliqa.com/whitepaper.pdf (2017).[92] V. Buterin, Ethereum sharding faq, https://github.com/ethereum/wiki/wiki/Sharding-FAQ (2017).[93] T. Nguyen-Van, T. Nguyen-Anh, T. Le, M. Nguyen-Ho, T. Nguyen-Van, N. Le, K. Nguyen-An, Scalable distributed ran-dom number generation based on homomorphic encryption, in: IEEE International Conference on Blockchain, Blockchain2019, Atlanta, GA, USA, July 14-17, 2019, 2019, pp. 572–579.[94] L. Zhang, H. Kan, Z. Chen, Z. Mao, J. Gao, Aberand: Effective distributed randomness on ciphertext-policy attribute-based encryption, https://eprint.iacr.org/2019/1307 (2019).[95] H. Dang, T. T. A. Dinh, D. Loghin, E. Chang, Q. Lin, B. C. Ooi, Towards scaling blockchain systems via sharding, in:Proceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019, Amsterdam, TheNetherlands, June 30 - July 5, 2019, 2019, pp. 123–140.[96] A. Hafid, A. S. Hafid, M. Samih, New mathematical model to analyze security of sharding-based blockchain protocols,IEEE Access 7 (2019) 185447–185457.[97] R. A. Bazzi, Synchronous byzantine quorum systems, Distributed Computing 13 (1) (2000) 45–52.
98] I. Abraham, D. Malkhi, K. Nayak, L. Ren, M. Yin, Sync hotstuff: Simple and practical synchronous state machinereplication, in: 2020 IEEE Symposium on Security and Privacy, SP 2020, San Francisco, CA, USA, May 18-21, 2020,2020, pp. 106–118.[99] A. Miller, Y. Xia, K. Croman, E. Shi, D. Song, The honey badger of BFT protocols, in: Proceedings of the 2016 ACMSIGSAC Conference on Computer and Communications Security, Vienna, Austria, October 24-28, 2016, 2016, pp. 31–42.[100] L. Lamport, The part-time parliament, ACM Trans. Comput. Syst. 16 (2) (1998) 133–169.[101] M. Yin, D. Malkhi, M. K. Reiter, G. Golan-Gueta, I. Abraham, Hotstuff: BFT consensus with linearity and responsive-ness, in: Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, PODC 2019, Toronto, ON,Canada, July 29 - August 2, 2019, 2019, pp. 347–356.[102] M. Al-Bassam, A. Sonnino, S. Bano, D. Hrycyszyn, G. Danezis, Chainspace: A sharded smart contracts platform, in:25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February18-21, 2018, 2018, pp. 18–21.[103] T. H. Chan, R. Pass, E. Shi, Pala: A simple partially synchronous blockchain, https://eprint.iacr.org/2018/981.pdf (2018).[104] C. Huang, Z. Wang, H. Chen, Q. Hu, Q. Zhang, W. Wang, X. Guan, Repchain: A reputation based secure, fast and highincentive blockchain system via sharding, IEEE Internet of Things Journal (2020) 1–1.[105] M. Zhang, J. Li, Z. Chen, H. Chen, X. Deng, Cycledger: A scalable and secure parallel protocol for distributed ledgervia sharding, in: 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS), New Orleans, LA,USA, May 18-22, 2020, 2020, pp. 358–367.[106] M. Wang, Q. Wu, Lever: Breaking the shackles of scalable on-chain validation, https://eprint.iacr.org/2019/1172 (2019).[107] G. Danezis, S. Meiklejohn, Centrally banked cryptocurrencies, in: 23rd Annual Network and Distributed System SecuritySymposium, NDSS 2016, San Diego, California, USA, February 21-24, 2016, 2016.[108] G. Avarikioti, E. Kokoris-Kogias, R. Wattenhofer, Divide and scale: Formalization of distributed ledger sharding proto-cols, http://arxiv.org/abs/1910.10434 (2019).[109] T. Rajab, M. H. Manshaei, M. Dakhilalian, M. Jadliwala, M. A. Rahman, On the feasibility of sybil attacks in shard-basedpermissionless blockchains, https://arxiv.org/abs/2002.06531 (2020).[110] Y. Liu, J. Liu, Z. Zhang, H. Yu, A fair selection protocol for committee-based permissionless blockchains, Computers &Security (2020) 101718.[111] R. Pass, E. Shi, Thunderella: Block with optimistic instant confirmation, in: Advances in Cryptology - EUROCRYPT2018 - 37th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Tel Aviv,Israel, April 29 - May 3, 2018 Proceedings, Part II, 2018, pp. 3–33.[112] B. David, P. Gazi, A. Kiayias, A. Russell, Ouroboros praos: An adaptively-secure, semi-synchronous proof-of-stakeblockchain, in: Advances in Cryptology - EUROCRYPT 2018 - 37th Annual International Conference on the Theoryand Applications of Cryptographic Techniques, Tel Aviv, Israel, April 29 - May 3, 2018 Proceedings, Part II, 2018, pp.66–98.[113] I. Eyal, E. G. Sirer, Majority is not enough: bitcoin mining is vulnerable, Commun. ACM 61 (7) (2018) 95–102.[114] I. Eyal, The miner’s dilemma, in: 2015 IEEE Symposium on Security and Privacy, SP 2015, San Jose, CA, USA, May17-21, 2015, 2015, pp. 89–103.[115] K. Nayak, S. Kumar, A. Miller, E. Shi, Stubborn mining: Generalizing selfish mining and combining with an eclipseattack, in: EuroS&P 2016, 2016, pp. 305–320.[116] Y. Liu, Y. Hei, T. Xu, J. Liu, An evaluation of uncle block mechanism effect on ethereum selfish and stubborn miningcombined with an eclipse attack, IEEE Access 8 (2020) 17489–17499.[117] S. Bag, S. Ruj, K. Sakurai, Bitcoin block withholding attack: Analysis and mitigation, IEEE Trans. Information Forensicsand Security 12 (8) (2017) 1967–1978.[118] Y. Kwon, D. Kim, Y. Son, E. Y. Vasserman, Y. Kim, Be selfish and avoid dilemmas: Fork after withholding (FAW)attacks on bitcoin, in: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security,CCS 2017, Dallas, TX, USA, October 30 - November 03, 2017, 2017, pp. 195–209.[119] E. Heilman, A. Kendler, A. Zohar, S. Goldberg, Eclipse attacks on bitcoin’s peer-to-peer network, in: 24th USENIXSecurity Symposium, USENIX Security 15, Washington, D.C., USA, August 12-14, 2015., 2015, pp. 129–144.[120] Y. Marcus, E. Heilman, S. Goldberg, Low-resource eclipse attacks on ethereum’s peer-to-peer network, http://eprint.iacr.org/2018/236 ftp://ftp.gate.cnrs.fr/RePEc/2014/1404.pdf (2014).[125] A. Chepurnoy, Interactive proof-of-stake, http://arxiv.org/abs/1601.00275 (2016).[126] E. Syta, P. Jovanovic, E. Kokoris-Kogias, N. Gailly, L. Gasser, I. Khoffi, M. J. Fischer, B. Ford, Scalable bias-resistantdistributed randomness, in: 2017 IEEE Symposium on Security and Privacy, SP 2017, San Jose, CA, USA, May 22-26,2017, 2017, pp. 444–460.[127] J. Kwon, Tendermint: Consensus without mining, https://tendermint.com/static/docs/tendermint.pdf (2014). https://eprint.iacr.org/2020/096 (2020).[142] R. Gennaro, S. Jarecki, H. Krawczyk, T. Rabin, Secure distributed key generation for discrete-log based cryptosystems,J. Cryptology 20 (1) (2007) 51–83.[143] C. Cachin, K. Kursawe, V. Shoup, Random oracles in constantinople: Practical asynchronous byzantine agreement usingcryptography, J. Cryptology 18 (3) (2005) 219–246.[144] A. Shamir, How to share a secret, Commun. ACM 22 (11) (1979) 612–613.[145] P. Feldman, A practical scheme for non-interactive verifiable secret sharing, in: 28th Annual Symposium on Foundationsof Computer Science, Los Angeles, California, USA, 27-29 October 1987, 1987, pp. 427–437.[146] M. Blum, A. D. Santis, S. Micali, G. Persiano, Noninteractive zero-knowledge, SIAM J. Comput. 20 (6) (1991) 1084–1118.[147] I. Cascudo, B. David, ALBATROSS: publicly attestable batched randomness based on secret sharing, in: Advancesin Cryptology - ASIACRYPT 2020, 268th International Conference on the Theory and Application of Cryptology andInformation Security, Virtual, December 7-11, 2020, Proceedings, 2020.[148] E. Syta, I. Tamas, D. Visher, D. I. Wolinsky, P. Jovanovic, L. Gasser, N. Gailly, I. Khoffi, B. Ford, Keeping authorities”honest or bust” with decentralized witness cosigning, in: IEEE Symposium on Security and Privacy, SP 2016, San Jose,CA, USA, May 22-26, 2016, 2016, pp. 526–545.[149] R. Canetti, Universally composable security: A new paradigm for cryptographic protocols, in: 42nd Annual Symposiumon Foundations of Computer Science, FOCS 2001, 14-17 October 2001, Las Vegas, Nevada, USA, 2001, pp. 136–145.[150] B. B¨unz, S. Goldfeder, J. Bonneau, Proofs-of-delay and randomness beacons in ethereum, IEEE Security and Privacy onthe blockchain (IEEE S&B) (2017).[151] S. Azouvi, P. McCorry, S. Meiklejohn, Winning the caucus race: Continuous leader election via public randomness, http://arxiv.org/abs/1801.07965 (2018).[152] A. K. Lenstra, B. Wesolowski, A random zoo: sloth, unicorn, and trx, http://eprint.iacr.org/2015/366 (2015).[153] P. Schindler, A. Judmayer, M. Hittmeir, N. Stifter, E. R. Weippl, Randrunner: Distributed randomness from trapdoorvdfs with strong uniqueness, https://eprint.iacr.org/2020/942.pdf (2020).[154] randao.org, Randao: Verifiable random number generation, https://randao.org/whitepaper/Randao_v0.85_en.pdf (2017).[155] B. Cohen, K. Pietrzak, The chia network blockchain, (2019).[156] R. Cramer, I. Damg˚ard, J. B. Nielsen, Multiparty computation from threshold homomorphic encryption, in: Advances inCryptology - EUROCRYPT 2001, International Conference on the Theory and Application of Cryptographic Techniques,Innsbruck, Austria, May 6-10, 2001, Proceeding, 2001, pp. 280–299.[157] Y. Rouselakis, B. Waters, Efficient statically-secure large-universe multi-authority attribute-based encryption, in: Finan-cial Cryptography and Data Security - 19th International Conference, FC 2015, San Juan, Puerto Rico, January 26-30,2015, Revised Selected Papers, 2015, pp. 315–332.[158] A. Cherniaeva, I. Shirobokov, O. Shlomovits, Homomorphic encryption random beacon, https://eprint.iacr.org/ (2019).[159] W. Hoeffding, Probability inequalities for sums of bounded random variables, in: The Collected Works of WassilyHoeffding, Springer, 1994, pp. 409–426.[160] S. Li, M. Yu, C. Yang, A. S. Avestimehr, S. Kannan, P. Viswanath, Polyshard: Coded sharding achieves linearly scalingefficiency and security simultaneously, IEEE Trans. Inf. Forensics Secur. 16 (2021) 249–261.[161] Q. Yu, S. Li, N. Raviv, S. M. M. Kalan, M. Soltanolkotabi, A. S. Avestimehr, Lagrange coded computing: Optimaldesign for resiliency, security, and privacy, in: The 22nd International Conference on Artificial Intelligence and Statistics,AISTATS 2019, 16-18 April 2019, Naha, Okinawa, Japan, 2019, pp. 1215–1225.[162] S. Liu, P. Viotti, C. Cachin, V. Qu´ema, M. Vukolic, XFT: practical fault tolerance beyond crashes, in: 12th USENIXSymposium on Operating Systems Design and Implementation, OSDI 2016, Savannah, GA, USA, November 2-4, 2016.,2016, pp. 485–500.[163] I. Abraham, S. Devadas, K. Nayak, L. Ren, Brief announcement: Practical synchronous byzantine consensus, in: 31stInternational Symposium on Distributed Computing, DISC 2017, October 16-20, 2017, Vienna, Austria, 2017, pp. 41:1–41:4.[164] A. Kiayias, A. Russell, Ouroboros-bft: A simple byzantine fault tolerant consensus protocol, https://eprint.iacr.org/2018/1049.pdf (2018).[165] D. Malkhi, K. Nayak, L. Ren, Flexible byzantine fault tolerance, in: Proceedings of the 2019 ACM SIGSAC Conferenceon Computer and Communications Security, CCS 2019, London, UK, November 11-15, 2019, 2019, pp. 1041–1053.[166] T. H. Chan, R. Pass, E. Shi, Pili: An extremely simple synchronous blockchain, https://eprint.iacr.org/2018/980 (2018).[167] C. Burchert, R. Wattenhofer, pichain: When a blockchain meets paxos, in: 21st International Conference on Principlesof Distributed Systems, OPODIS 2017, Lisbon, Portugal, December 18-20, 2017, 2017, pp. 2:1–2:13.[168] A. Charapko, A. Ailijiang, M. Demirbas, Bridging paxos and blockchain consensus, in: IEEE International Conference onInternet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physicaland Social Computing (CPSCom) and IEEE Smart Data (SmartData), iThings/GreenCom/CPSCom/SmartData 2018,Halifax, NS, Canada, July 30 - August 3, 2018, 2018, pp. 1545–1552.[169] D. J. Bernstein, The poly1305-aes message-authentication code, in: Fast Software Encryption: 12th International Work-shop, FSE 2005, Paris, France, February 21-23, 2005, Revised Selected Papers, 2005, pp. 32–49.[170] M. Yin, D. Malkhi, M. K. Reiter, G. G. Gueta, I. Abraham, Hotstuff: Bft consensus with linearity and responsiveness, in:Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, PODC 2019, Toronto, ON, Canada,July 29 - August 2, 2019, ACM, 2019, pp. 347–356.[171] G. G. Gueta, I. Abraham, S. Grossman, D. Malkhi, B. Pinkas, M. Reiter, D.-A. Seredinschi, O. Tamir, A. Tomescu,Sbft: a scalable and decentralized trust infrastructure, in: 2019 49th Annual IEEE/IFIP international conference ondependable systems and networks (DSN), IEEE, 2019, pp. 568–580.[172] C. Schnorr, Efficient signature generation by smart cards, J. Cryptology 4 (3) (1991) 161–174.[173] J. Sousa, A. N. Bessani, From byzantine consensus to BFT state machine replication: A latency-optimal transformation,in: 2012 Ninth European Dependable Computing Conference, Sibiu, Romania, May 8-11, 2012, 2012, pp. 37–48.[174] M. J. Fischer, N. A. Lynch, M. Paterson, Impossibility of distributed consensus with one faulty process, J. ACM 32 (2)(1985) 374–382.[175] C. Cachin, K. Kursawe, F. Petzold, V. Shoup, Secure and efficient asynchronous broadcast protocols, in: Advances inCryptology - CRYPTO 2001, 21st Annual International Cryptology Conference, Santa Barbara, California, USA, August19-23, 2001, Proceedings, 2001, pp. 524–541.[176] M. O. Rabin, Randomized byzantine generals, in: 24th Annual Symposium on Foundations of Computer Science, Tucson,Arizona, USA, 7-9 November 1983, 1983, pp. 403–409.[177] M. Ben-Or, Another advantage of free choice: Completely asynchronous agreement protocols (extended abstract), in:Proceedings of the Second Annual ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing, Montreal,Quebec, Canada, August 17-19, 1983, 1983, pp. 27–30.[178] G. S. Veronese, M. Correia, A. N. Bessani, L. C. Lung, P. Ver´ıssimo, Efficient byzantine fault-tolerance, IEEE Trans.Computers 62 (1) (2013) 16–30.[179] I. Abraham, D. Malkhi, A. Spiegelman, Asymptotically optimal validated asynchronous byzantine agreement, in: Pro-ceedings of the 2019 ACM Symposium on Principles of Distributed Computing, PODC 2019, Toronto, ON, Canada, July29 - August 2, 2019, 2019, pp. 337–346.[180] S. Duan, M. K. Reiter, H. Zhang, BEAT: asynchronous BFT made practical, in: Proceedings of the 2018 ACM SIGSACConference on Computer and Communications Security, CCS 2018, Toronto, ON, Canada, October 15-19, 2018, 2018,pp. 2028–2041.[181] Y. Lu, Z. Lu, Q. Tang, G. Wang, Dumbo-mvba: Optimal multi-valued validated asynchronous byzantine agreement,revisited, in: PODC ’20: ACM Symposium on Principles of Distributed Computing, Virtual Event, Italy, August 3-7,2020, 2020, pp. 129–138.[182] B. Guo, Z. Lu, Q. Tang, J. Xu, Z. Zhang, Dumbo: Faster asynchronous BFT protocols, in: Proceedings of the 2020ACM SIGSAC Conference on Computer and Communications Security, CCS 2020, Virtual, November 9-13, 2020, 2020.[183] Y. Liu, J. Liu, J. Yin, G. Li, H. Yu, Q. Wu, Cross-shard transaction processing in sharding blockchains, in: Algorithmsand Architectures for Parallel Processing - 20th International Conference, ICA3PP 2020, New York City, NY, USA,October 2-4, 2020, Proceedings, Part III, 2020, pp. 324–339.[184] Y. Liu, J. Liu, D. Li, H. Yu, Q. Wu, Fleetchain: A secure scalable and responsive blockchain achieving optimal sharding,in: Algorithms and Architectures for Parallel Processing - 20th International Conference, ICA3PP 2020, New York City, Y, USA, October 2-4, 2020, Proceedings, Part III, 2020, pp. 409–425.[185] D. Currin, J. Denman, L. G. M. Eykholt, Mobile process calculi for programming the blockchain documentation, https://buildmedia.readthedocs.org/media/pdf/mytestdocforrchain/latest/mytestdocforrchain.pdf (2017).[186] V. Buterin, Cross-shard contract yanking, https://ethresear.ch/t/cross-shard-contract-yanking/1450 (2018).[187] A. Manuskin, M. Mirkin, I. Eyal, Ostraka: Secure blockchain scaling by node sharding, in: IEEE European Symposiumon Security and Privacy Workshops, EuroS&P Workshops 2020, Genoa, Italy, September 7-11, 2020, 2020, pp. 397–406.[188] A. Sonnino, S. Bano, M. Al-Bassam, G. Danezis, Replay attacks and defenses against cross-shard consensus in shardeddistributed ledgers, in: IEEE European Symposium on Security and Privacy Workshops, EuroS&P Workshops 2020,Genoa, Italy, September 7-11, 2020, 2020, pp. 397–406.[189] L. N. Nguyen, T. D. T. Nguyen, T. N. Dinh, M. T. Thai, Optchain: Optimal transactions placement for scalableblockchain sharding, in: 39th IEEE International Conference on Distributed Computing Systems, ICDCS 2019, Dallas,TX, USA, July 7-10, 2019, 2019, pp. 525–535.[190] H. Dang, A. Dinh, E. Chang, B. C. Ooi, Chain of trust: Can trusted hardware help scaling blockchains?, http://arxiv.org/abs/1804.00399 (2018).[191] D. Leung, A. Suhl, Y. Gilad, N. Zeldovich, Vault: Fast bootstrapping for the algorand cryptocurrency, in: 26th AnnualNetwork and Distributed System Security Symposium, NDSS 2019, San Diego, California, USA, February 24-27, 2019,2019.[192] M. Andrychowicz, S. Dziembowski, Pow-based distributed cryptography with no trusted setup, in: Advances in Cryptol-ogy - CRYPTO 2015 - 35th Annual Cryptology Conference, Santa Barbara, CA, USA, August 16-20, 2015, Proceedings,Part II, 2015, pp. 379–399.[193] J. A. Garay, A. Kiayias, N. Leonardos, G. Panagiotakos, Bootstrapping the blockchain, with applications to consensusand fast PKI setup, in: Public-Key Cryptography - PKC 2018 - 21st IACR International Conference on Practice andTheory of Public-Key Cryptography, Rio de Janeiro, Brazil, March 25-29, 2018, Proceedings, Part II, 2018, pp. 465–495.[194] M. H. Manshaei, M. Jadliwala, A. Maiti, M. Fooladgar, A game-theoretic analysis of shard-based permissionless block-chains, IEEE Access 6 (2018) 78100–78112.[195] S. D. Kamvar, M. T. Schlosser, H. Garcia-Molina, The eigentrust algorithm for reputation management in P2P networks,in: Proceedings of the Twelfth International World Wide Web Conference, WWW 2003, Budapest, Hungary, May 20-24,2003, 2003, pp. 640–651.[196] M. Nojoumian, A. Golchubian, L. Njilla, K. Kwiat, C. Kamhoua, Incentivizing blockchain miners to avoid dishonestmining strategies by a reputation-based paradigm, in: Science and Information Conference, 2018, pp. 1118–1134.[197] A. Bugday, A. Ozsoy, H. Sever, Securing blockchain shards by using learning based reputation and verifiable randomfunctions, in: 2019 International Symposium on Networks, Computers and Communications, ISNCC 2019, Istanbul,Turkey, June 18-20, 2019, 2019, pp. 1–4.[198] E. Maskin, Nash equilibrium and welfare optimality, The Review of Economic Studies 66 (1) (1999) 23–38.[199] G. Yu, X. Wang, K. Yu, W. Ni, J. A. Zhang, R. P. Liu, Survey: Sharding in blockchains, IEEE Access 8 (2020)14155–14181.[200] R. Han, J. Yu, R. Zhang, Analysing and improving shard allocation protocols for sharded blockchains, https://eprint.iacr.org/2020/943 (2020).[201] S. Bano, M. Al-Bassam, G. Danezis, The road to scalable blockchain designs, login Usenix Mag. 42 (4) (2017).[202] J. A. Garay, A. Kiayias, Sok: A consensus taxonomy in the blockchain era, in: Topics in Cryptology - CT-RSA 2020 -The Cryptographers’ Track at the RSA Conference 2020, San Francisco, CA, USA, February 24-28, 2020, Proceedings,2020, pp. 284–318.[203] S. J. Alsunaidi, F. A. Alhaidari, A survey of consensus algorithms for blockchain technology, in: 2019 InternationalConference on Computer and Information Sciences (ICCIS), IEEE, 2019, pp. 1–6.[204] G. Nguyen, K. Kim, A survey about consensus algorithms used in blockchain, J. Inf. Process. Syst. 14 (1) (2018) 101–128.[205] Y. Xiao, N. Zhang, W. Lou, Y. T. Hou, A survey of distributed consensus protocols for blockchain networks, IEEECommunications Surveys and Tutorials 22 (2) (2020) 1432–1465.[206] J. Xie, F. R. Yu, T. Huang, R. Xie, J. Liu, Y. Liu, A survey on the scalability of blockchain systems, IEEE Network33 (5) (2019) 166–173.[207] S. Kim, Y. Kwon, S. Cho, A survey of scalability solutions on blockchain, in: International Conference on Informationand Communication Technology Convergence, ICTC 2018, Jeju Island, Korea (South), October 17-19, 2018, 2018, pp.1204–1207.[208] Q. Zhou, H. Huang, Z. Zheng, J. Bian, Solutions to scalability of blockchain: A survey, IEEE Access 8 (2020) 16440–16455.(2020).[201] S. Bano, M. Al-Bassam, G. Danezis, The road to scalable blockchain designs, login Usenix Mag. 42 (4) (2017).[202] J. A. Garay, A. Kiayias, Sok: A consensus taxonomy in the blockchain era, in: Topics in Cryptology - CT-RSA 2020 -The Cryptographers’ Track at the RSA Conference 2020, San Francisco, CA, USA, February 24-28, 2020, Proceedings,2020, pp. 284–318.[203] S. J. Alsunaidi, F. A. Alhaidari, A survey of consensus algorithms for blockchain technology, in: 2019 InternationalConference on Computer and Information Sciences (ICCIS), IEEE, 2019, pp. 1–6.[204] G. Nguyen, K. Kim, A survey about consensus algorithms used in blockchain, J. Inf. Process. Syst. 14 (1) (2018) 101–128.[205] Y. Xiao, N. Zhang, W. Lou, Y. T. Hou, A survey of distributed consensus protocols for blockchain networks, IEEECommunications Surveys and Tutorials 22 (2) (2020) 1432–1465.[206] J. Xie, F. R. Yu, T. Huang, R. Xie, J. Liu, Y. Liu, A survey on the scalability of blockchain systems, IEEE Network33 (5) (2019) 166–173.[207] S. Kim, Y. Kwon, S. Cho, A survey of scalability solutions on blockchain, in: International Conference on Informationand Communication Technology Convergence, ICTC 2018, Jeju Island, Korea (South), October 17-19, 2018, 2018, pp.1204–1207.[208] Q. Zhou, H. Huang, Z. Zheng, J. Bian, Solutions to scalability of blockchain: A survey, IEEE Access 8 (2020) 16440–16455.