aa r X i v : . [ c s . D B ] F e b Distributed Cross Blockchain Transactions
Dongfang Zhao
University of Nevada, Reno [email protected] Tonglin Li
Lawrence Berkeley National Laboratory [email protected]
ABSTRACT
The interoperability across multiple or many blockchainswould play a critical role in the forthcoming blockchain-based data management paradigm. In particular, how toensure the ACID properties of those transactions across anarbitrary number of blockchains remains an open problem inboth academic and industry: Existing solutions either workfor only two blockchains or requires a centralized compo-nent, neither of which would meet the scalability require-ment in practice. This short paper shares our vision andsome early results toward scalable cross-blockchain transac-tions. Specifically, we design two distributed commit proto-cols and, both analytically and experimentally, demonstratetheir effectiveness.
PVLDB Reference Format:
Dongfang Zhao and Tonglin Li. Distributed Cross-BlockchainTransactions.
PVLDB , 12(xxx): xxxx-yyyy, 2020.DOI: https://doi.org/10.14778/xxxxxxx.xxxxxxx
1. INTRODUCTION1.1 Motivation
A blockchain offers an immutable, decentralized, and anony-mous mechanism for transactions between two entities onthe same blockchain. Blockchain was not originally designedfor online transactional processing (OLTP) workloads; in-stead, it aimed to offer an autonomous and tamper-proofledger service among mutually-distrusted parties and there-fore, early blockchain systems can deliver only mediocretransaction throughput.One natural question is whether and how we can adoptblockchains to efficiently handle OLTP workloads such thatboth autonomy and performance can be achieved at thesame time. Indeed, much recent work focuses on this direc-tion: in [11, 13], authors advocate to leverage blockchains forOLTP workloads with various optimizations (e.g., sharding[7], sidechains [12]) to boost up the transaction throughputof blockchains, such that blockchains would deliver similarly
This work is licensed under the Creative Commons Attribution NonCommercial NoDerivatives 4.0 International License. To view a copyof this license, visit http://creativecommons.org/licenses/by nc nd/4.0/. Forany use beyond those covered by this license, obtain permission by [email protected]. Copyright is held by the owner/author(s). Publication rightslicensed to the VLDB Endowment.
Proceedings of the VLDB Endowment,
Vol. 12, No. xxxISSN 2150 8097.DOI: https://doi.org/10.14778/xxxxxxx.xxxxxxx high performance as relational database systems (RDBMS)and turn to be a competitive alternative to the latter as ageneral-purpose data management system.There is yet another critical, often overlooked, issue thatmust be addressed before blockchains can be widely adoptedas a general data management paradigm: the interoperabil-ity across heterogeneous blockchains. While SQL along withthe underlying distributed transaction handling are avail-able between different vendors’ RDBMS implementations,no such interface or general mechanism exists for blockchains.Recent attempts (e.g., Cosmos [6]) on such cross-blockchaintransactions are all ad hoc and exhibits poor scalability dueto the centralized (physical or virtual) broker.
We list four outstanding limitations exhibited by state-of-the-art cross-blockchain solutions: (1) Centralized Broker.
The transactions between het-erogeneous blockchains are managed by a third-party, usu-ally implemented as another blockchain (it is called a hub in Cosmos). This is against the decentralization principleof blockchains: the broker would become a performancebottleneck, a single-point-of-failure, a target of security at-tacks. Similarly, a recent work called AC3 [16] employs anextra component (known as witness blockchain ) as a cen-tral authority to govern the cross-chain operations. Al-though the witness blockchain is comprised of the nodesfrom existing blockchains, still, these virtual nodes on thewitness blockchain become the critical components of theentire ecosystem and, again, break the very core principle ofblockchains. (2) Two-Party Transactions.
The protocols used byexisting cross-blockchain systems stem from the sidechainprotocol [12], which was originally designed for transferringassets between Bitcoin [3] and another cryptocurrency. Thesidechain protocol speaks of nothing about three- or multi-party transactions; in fact, Cosmos only supports transfer-ring assets between Bitcoin [3] and Ethereum [8]. A morerecent line of works [10, 9] are based on two-party AtomicCross-Chain Swaps (ACS); however, ACS cannot guaranteethe atomicity of the multi-blockchain transaction as a whole. (3) Performance.
The sidechain protocol [12] took hours,if not days, to commit a single cross-blockchain transaction.The main reason for this is due to the possible branchesfrom the participating blockchains. In any participatingblockchain, only one (i.e., the longest one) branch will re-main valid, and any transactions from the shorter brancheswill rollback. This is not a problem if all of the transac-1ion parties are from the same blockchain; But for cross-blockchain transactions, deliberate actions need to be taken. (4) Conventional Distributed Transactions.
Onecould argue that why not applying existing approaches ofdistributed transactions to multi-party blockchains? Theshort answer is that the conventional wisdom did not as-sume the participant to proactively “rollback” its own deci-sion, which is not uncommon in blockchains. For instance,in the conventional 2PC protocol [1], when a participantreplies a ready-to-commit message to the coordinator, weassume that the decision is final and we can proceed to thenext phase of the protocol. In blockchains, however, the ready-to-commit message can be revoked by the partici-pant later on, even after the transaction is completed onlybecause the transaction happens to reside on a branch thatis suppressed by a longer branch. There was little study onsuch “regrettable” behavior of blockchains in the literatureof distributed transactions.
For completeness, Table 1 summarizes candidate solutionswith respect to two important properties regarding cross-blockchain transactions. As we can see, existing works arelimited to centralized design (i.e., the requirement of a hub),or the potential blocking, or both. In our prior work [17],we presented the roadmap toward cross-blockchain transac-tions, named CBT, to overcome the above limitations.
Table 1: Popular Cross-Blockchain Transaction Pro-tocols.
Blocking NonblockingCentralized Sidechain [12] AC3 [16]Distributed 2PC [1] CBT [17]This paper is the first step toward the goals proposedin [17]. Specifically, we will present a set of nonblockingdistributed commit protocols designed for multi-party cross-blockchain transactions ( § § §
2. PROTOCOLS2.1 System Models and Assumptions
We assume the nodes follow a crash failure model. Thatis, there are no arbitrary failures from the underlying blockchainsand their participants. We make this strong assumption asa starting point for this direction of research; a
Byzantinefailure model will be discussed in the future work. Fur-thermore, we assume the crashed node will eventually berecovered and can be replaced by a functional node in areasonable time, denoted by f . Moreover, during a singletransaction, the failures will not happen indefinitely but forfinite times denoted by λ .We assume the network transfer can be delayed but notindefinitely: the communication is asynchronous and persis-tent. That is, the messages can be eventually delivered in areasonable time. The latency of the network is denoted by τ . We assume that a blockchain can finalize the main branchin finite time, after which the transactions cannot be rolled back. In Bitcoin, for example, the pending time is about onehour—six blocks of transactions. We denote the averagepending time for C i is δ i , which also includes the waiting time for a transaction to be picked up by the system.We assume there is an effective programmable way for dif-ferent blockchains to communicate. This is mostly true fornew blockchain implementations with smart contracts . Forthose old systems, e.g., Bitcoin, that do not support smartcontract, we assume a proxy is available on such systems forthe cross-blockchain communications. Notations.
We denote the set of blockchains as C = { C i } , where each C i , i ∈ Z + , represents a specific blockchainin the consortium of blockchains. We use C − i to denote thecomplement set C \{ C i } , following the naming convention ingame theory. The cardinality, or order, of the set, i.e., |C| ,indicates the total number of blockchains involved in thetransaction. Each blockchain C i comprises a series of linkedblocks, denoted as B ji , where the superscript j indicatesthe index of the block on blockchain C i . Each block is filledwith a series of transactions, denoted by T k , where k impliesa universally unique identifier (UUID) of each transactionsince the inception of the blockchain consortium. It shouldbe clear that, however, although k is unique globally, it willappear at least once on each C i and possibly more thanonce if C i has branches during the processing of T k . Foreach C i , there is a corresponding set N i ⊆ N denoting theset of nodes having joined the network of blockchain C i . Itis possible that a node joining multiple blockchains: n ∈ N i and n ∈ N j , i = j . Metrics.
Throughput is, arguably, the most popular met-ric in evaluating the performance of blockchains. As in manyother areas, the throughput of transactions is defined as thenumber of transactions completed in a time unit, usually ina second. What is less used, or somewhat overlooked, met-ric, is the latency of a transaction, measuring the lifespanof a single transaction in the blockchain systems. We ar-gue that latency is a more interesting metric from a user’sstandpoint: she cares more about when her transaction iscompleted than how many concurrent transactions can behandled by the system per se , concerned with by the systemadmin.
The first protocol is called Synchronous cross-Blockchaintransactions Protocol (SBP) that is designed to strictly en-force the ACID properties of cross-blockchain transactions.The targeting workloads include those that need to followstrong consistency models such as financial transactions. Asa trade-off, the performance, especially the latency, is notat the high end of the spectrum of candidate protocols.SBP respects each individual blockchain’s own branchesand delays the global commit until no single blockchaincan unilaterally rollback the transaction. As the conven-tional wisdom in distributed commit protocols, a specificblockchain initiates the multi-party transaction. In the liter-ature, this initiator is usually called a coordinator , althoughwe want to point out that this coordinator can be any par-ticipant C i in the pool C . Many leader election algorithmscan be applied to select the coordinator with the proxies on C ’s. The specific node n ∈ N i serving as the endpoint for the inter-blockchain communication can also be arbitrar-ily selected as long as the following conditions are met: (i)2ther nodes N i \ { n } are aware of the role of n and (ii) allthe intra-blockchain transaction updates have been appliedto n .Suppose C i initiates a transaction T k among all elementsin C , and |C| ≥
3. We will start describing the protocol inthe civil case . Phase I
First, C i broadcasts a precommit message to (theproxies of) C . It should be clear that C i in this caseserves as both the coordinator and a (local) partic-ipant . C i then waits for a Ready reply from eachblockchain in C . A blockchain C j ∈ C (again, j = i is allowed, implying a local message) replies a ready message to C i after (i) all prerequisites are satisfied,e.g., the balance is higher than the funds to be de-ducted in a cryptocurrency application, and (ii) moreimportantly, the entity is locked . The second action iscrucial to avoid double-spending issues. Phase II
Second, C i braodcasts a commit message to C andwaits for a done reply from each element in C . Ablockchain C j ∈ C carries out its local operation, andwait for δ j before returning a done message to C i . Theparticipants then should unlock the entities. Once C i receives |C| done replies, T k is marked completed.Therefore, the civil case of SBP runs much like a 2PCprotocol except for the introduction of pending time δ j .The period enforced by δ j can only preclude the possiblebranches in blockchains, and yet cannot avoid the possibleblocking in the uncivil case where nodes do fail (up to crashfailures) incur possible blocking. One way to fix that is tointroduce an additional phase, essentially extending the pro-tocol into three phases, which has been extensively studiedin the literature and is not a practical approach due to un-acceptable performance. What we propose to overcome theblocking issue is more lightweight: taking a passive heart-beat approach to effectively detect node failures. It shouldbe noted that this approach becomes effective only becausein cross-blockchain transactions each node is essentially a setof nodes, i.e., N i for C i , such that if the original proxy node n ∈ N i fails, we can quickly re-select n ′ ∈ N i to continuethe SBP protocol.Formally, suppose n ∈ N i is the endpoint of C i , the prox-ies on other nodes N i \{ n } run a heartbeat probe to n , whoseinterval is denoted as σ i . Let σ = sup { σ i , ≤ i ≤ |C|} , itis not hard to see that SBP can be blocked by up to σ . Inpractice, we can set σ ≪ δ , where δ = inf { δ j , ≤ j ≤ |C|} ,such that the heartbeat overhead is negligible. Atomicity.
SBP takes a conservative approach to com-mit the requested transaction. At any point during thetwo-phase protocol, any states other than the expected onesmentioned in the protocol narrative results in a global abort .A more subtle yet rare case is that no qualified node can befound after the heartbeat protocol detects a crash failure, inwhich case the entire SBP also aborts the transaction.
Consistency.
The changes incurred by the transactionwould be invisible to users until the C i marks the completionof the transaction. Thus, SBP implements a strong consis-tency model, there are no dirty-write or repeated-read is-sues during the course of distributed transaction processing.Indeed, this strong consistency is attributed to the lock-ing approach with the price of suboptimal performance in transaction latency. We will speak more about performancein the complexity discussion shortly. Isolation.
This can be trivially verified by the fact thatlocking and unlocking are implemented correctly, as dis-cussed in the protocol.
Durability.
Updates are persisted on all the nodes ineach involved blockchain.
We will show that the number of messages is asymptoti-cally polynomial to the number of nodes among all blockchains.
Proposition 1 (Number of messages).
The total num-ber of messages passed is O ( λ | N | ) . Proof.
Obviously, the maximal number of messages aresent when the nodes are failed repeatedly for finite times,and the transaction eventually completes. It is crucial tonote that the failure can happen for limited times becauseotherwise, our assumption would not hold (cf. § C is inter-blockchain z }| { · λ · ( |C| −
1) + intra-blockchain z }| {X C i ∈C ( | N i | − · λ · ( |C| −
1) + | N | − |C| = | N | + (2 λ − |C| − λ ≤ λ | N | . ( since |C| ≤ | N | and λ ≥ λ | N | , provingthe proposition.Thus, the number of messages is asymptotically polyno-mial to the number of nodes among all blockchains. Wethen study the theoretical upper bound of the transactionlatency. Proposition 2 (Latency upper bound).
The longestperiod for a single transaction, i.e., the latency, is boundedby τ + λ ( f + δ ) , where δ = sup { δ i , ≤ i ≤ |C|} . Proof.
The latency of phase I is calculated as∆ = 2 · τ + λ · f, and the latency of phase II is bounded by∆ = 2 · τ + λ · f + X n ∈ N i δ i | {z } λ , where n indicates a failed node and λ = λ + λ . Therefore,the overall latency∆ = ∆ + ∆ ≤ τ + ( λ + λ ) f + λ δ ≤ τ + λ ( f + δ ) . In practice, τ can be easily measured, in terms of mil-liseconds; f usually takes a few seconds, e.g., to reboot thefailed node; δ is also well understood: in Bitcoin, for in-stance, it takes hours to finalize a transaction. However, it3s not trivial to estimate λ other than keep an empirical logover the failure rate. We want to point out that a Poissondistribution can become a handy tool for quickly estimatingthe transaction delay. That is, the probability of k failurescan be estimated by λ k e − λ k ! , where e is Euler’s number. While SBP discussed in the previous section achieves strongconsistency, the price is the somewhat long delay. Therefore,SBP is ideal for those time-insensitive applications that arerequired to guarantee ACID properties. This section stud-ies the other end of the spectrum: what if the workloadis highly time-sensitive and can tolerate temporary incon-sistencies. That is, the applications, such as emails, canaccept an eventual consistency semantic. To this end, wedesign a distributed commit protocol, namely RBP, follow-ing the spirit of redo-logs that has been extensively studiedin databases.RBP makes a key change to the way how participants re-ply the done messages back to the coordinator C i . Insteadof waiting for a period of δ j , C j replies C i right after thelocal updates are completed. Indeed, the question then be-comes what if C j decides to cut off the branch comprisingthe completed transaction T k between C i and C j ’s later on?To this end, blockchain C i maintain a sliding window thatrecords recent transactions completed in the past δ i period.The rationale is that if any of these pending transactions areon the path of a shorter branch of C i , C i can take accord-ing actions such as returning the transactions back to therequest pool, or immediately rescheduling the transactions.RBP takes the former approach: transactions on the shorterbranches are recycled back into the pool of requests. Notethat we cannot construct complement transactions to undo the changes because those transactions are invisible to themain branch of C i .Evidently, RBP still meets the atomicity requirement:there is no “partial” transaction committed. It is also triv-ial to check that both isolation and durability hold in RBP.For consistency , RBP implements an eventual consistencysemantics: the transactions on shorter branches will eventu-ally be reprocessed. We conclude this section with a moredetailed quantitative study in the following.We are particularly interested in the improved latencypaid by the weak consistency semantics. Let the transac-tions in the sliding window of C i be T i . Consequently, thethroughput of blockchain C i can be calculated by |T i | δ i . Be-cause of the possible cascading effect implied by the C i ’sindeterministic branching behavior, we cannot derive an up-per bound over the latency of a transaction T k T i . However,if no branching happens during T k , the latency can be aslow as 4 τ + λf . Recall that both τ and f are orders of mag-nitude smaller than δ , and λ represents a few failed nodes ina time unit; therefore, RBP is expected to deliver a signifi-cantly smaller latency than SBP. Again, this gain is tradedby the (strong) consistency.
3. PRELIMINARY RESULTS
We have implemented RBP protocol as well as two base-line protocols, i.e., 2PC [1] and AC3 [16], on the BlockLitesystem [14]. The source code is accessible at Github [4]. The source code is written with Java of JDK 1.7. The code-base comprises about 5,190 lines of code. The codebase hasthree major components: (i) the blockchain implementationincluding protocols and utilities; (ii) the network componentincluding the communications among coordinator and par-ticipants; and (iii) the graphic user interface developed withJava Swing. A technical report on an earlier version of thesystem can be found at [15].The transaction data sets used for evaluation are ETC20and TPC-H. We feed up to three million transactions to thethree protocols (RBP, 2PC, and AC3) in a 64-blockchainenvironment. For 2PC, we set it up as the “ideal case”where no failures take place during the experiment; it is theupper-bound performance one can best expect from 2PC.The point is to show the overhead incurred by our proposedCBT compared to such upper-bound performance. For AC3,we arbitrarily select one blockchain as the “hub”, or “wit-ness blockchain” as in the literature. Because of AC3’scentralized hub, we expect the performance and scalabil-ity will be affected at some point, e.g., a larger number ofblockchains. Both 2PC and CBT show (almost) linear scala-bility because no centralized component exists in the system.Results show that RBP incurs insignificant overhead (com-pared with baseline 2PC) at small/medium scales: 3.6% –4% on 2–32 blockchains; then the overhead is negligible on64 blockchains. Compared with 2PC and RBP, AC3 startsto fall behind on eight blockchains due to its “hub” design.
4. CONCLUSION AND FUTURE WORK
As a concluding remark, we want to reemphasize that thefuture blockchain-based data management paradigm mustbe equipped with effective cross-blockchain transactions atarbitrary scales, which cannot be realized without a scal-able distributed commit protocol specifically designed fortransactional workloads, namely cross-blockchain transac-tion (CBT). The role of the CBT protocols for the futureblockchain-based paradigms can be considered as the ana-logue to: TCP/UDP for network systems, HTTP for webservers, FTP for file servers, and so forth. What is pre-sented in this short paper is only one of the first steps to-ward the future standardization when the time comes formany heterogeneous blockchain systems to jointly work asa coordinated service or platform.In addition to industry workloads represented by ETC20and TPC-H already tested with CBT, we are working with ateam at the Lawrence Berkeley National Laboratory on de-ploying CBT to one of the largest supercomputers Cori [5]to: (i) Justify the feasibility of blockchains for distributedcaching in high-performance computing systems; (ii) Evalu-ate the scalability and performance of CBT for huge scien-tific workloads (e.g., the data provenance of astronomy ap-plications); and (iii) Quantify the energy efficiency of large-scale blockchain deployment. Some preliminary results onhigh-performance blockchains can be found at [2].
Acknowledgement
This work is in part supported by the U.S. Department ofEnergy under contract number DE-SC0020455. This workis also supported by a Google Cloud award and an Amazonresearch award. The authors are grateful for the valuablediscussion with Mohammad Sadoghi (University of Califor-nia, Davis) on an earlier version of this work.4 . REFERENCES [1] 2PC Protocol. https://en.wikipedia.org/wiki/Two-phase_commit_protocol ,Accessed 2020.[2] A. Al-Mamun, T. Li, M. Sadoghi, L. Jiang, H. Shen,and D. Zhao. Hpchain: An mpi-based blockchainframework for data fidelity in high-performancecomputing systems. In
Proceedings of the InternationalConference on High Performance Computing,Networking, Storage and Analysis (SC) , 2019.[3] Bitcoin. https://bitcoin.org/bitcoin.pdf ,Accessed 2020.[4] CBT code repository. https://github.com/hpdic/cbt , Accessed 2020.[5] Cori. https://docs.nersc.gov/systems/cori ,Accessed 2020.[6] Cosmos Network. https://cosmos.network , Accessed2020.[7] H. Dang, T. T. A. Dinh, D. Loghin, E. Chang, Q. Lin,and B. C. Ooi. Towards scaling blockchain systems viasharding. In
SIGMOD , 2019.[8] Ethereum. , Accessed2020.[9] Herlihy and Maurice. Atomic cross-chain swaps. In
ACM Symposium on Principles of DistributedComputing (PODC) , 2018.[10] M. Herlihy, L. Shrira, and L. Shrira. Cross-chain dealsand adversarial commerce.
PVLDB , 13(2):100–113,2019.[11] E.-H. Muhammad, B. Carsten, A. Arvind, K. Donald,and R. Ravi. Blockchaindb: A shared database onblockchains.
Proc. VLDB Endow. , 12(11):1597–1609,July 2019.[12] Sidechains. https://blockstream.com/sidechains.pdf , Accessed2020.[13] J. Wang and H. Wang. Monoxide: Scale outblockchains with asynchronous consensus zones. In
USENIX Symposium on Networked Systems Designand Implementation (NSDI) , pages 95–112, 2019.[14] X. Wang, A. Al-Mamun, F. Yan, and D. Zhao. Towardaccurate and efficient emulation of public blockchainsin the cloud. In , pages 67–82, 2019.[15] X. Wang, O. T. Tawose, F. Yan, and D. Zhao.Distributed nonblocking commit protocols formany-party cross-blockchain transactions.
CoRR ,abs/2001.01174, 2020.[16] V. Zakhary, D. Agrawal, and A. El Abbadi. Atomiccommitment across blockchains.
CoRR ,abs/1905.02847, 2019.[17] D. Zhao. Cross-blockchain transactions. In