A Pub-Sub Architecture to Promote Blockchain Interoperability
Sara Ghaemi, Sara Rouhani, Rafael Belchior, Rui S. Cruz, Hamzeh Khazaei, Petr Musilek
AA Pub-Sub Architecture to Promote BlockchainInteroperability
Sara Ghaemi ∗ , Sara Rouhani † , Rafael Belchior ‡ , Rui S. Cruz ‡ , Hamzeh Khazaei § , Petr Musilek ∗∗ University of Alberta, Edmonton, Canada, { sghaemi, petr.musilek } @ualberta.ca † University of Saskatchewan, Saskatoon, Canada, [email protected] ‡ Instituto Superior T´ecnico, Universidade de Lisboa, Lisboa, Portugal, { rafael.belchior, rui.s.cruz } @tecnico.ulisboa.pt § York University, Toronto, Canada, [email protected]
Abstract —The maturing of blockchain technology leads toheterogeneity, where multiple solutions specialize in a particularuse case. While the development of different blockchain networksshows great potential for blockchains, the isolated networks haveled to data and asset silos, limiting the applications of thistechnology. Blockchain interoperability solutions are essentialto enable distributed ledgers to reach their full potential. Suchsolutions allow blockchains to support asset and data transfer,resulting in the development of innovative applications.This paper proposes a novel blockchain interoperability solu-tion for permissioned blockchains based on the publish/subscribearchitecture. We implemented a prototype of this platform toshow the feasibility of our design. We evaluate our solution byimplementing examples of the different publisher and subscribernetworks, such as Hyperledger Besu, which is an Ethereum client,and two different versions of Hyperledger Fabric. We presenta performance analysis of the whole network that indicates itslimits and bottlenecks. Finally, we discuss the extensibility andscalability of the platform in different scenarios. Our evaluationshows that our system can handle a throughput in the order ofthe hundreds of transactions per second.
Index Terms —Distributed Ledger Technology, Blockchain, In-teroperability, Publish/Subscribe
I. I
NTRODUCTION
The distributed ledger technology (DLT) enables a set ofindependent untrusted nodes to establish an agreement on thestate of a shared ledger. Blockchain, a type of DLT, is mostlyknown for its use cases in cryptocurrencies such as Bitcoin [1],Ethereum [2], and XRP [3], among others. However, thetechnology can be used for more diverse applications andindustries. Some examples are biomedical and health care [4],[5], Internet of Things (IoT) [6], [7], public administration [8],[9], and cloud computing [10], [11]. Since each industry has itsown unique sets of requirements, many isolated permissionedand permissionless blockchains have been introduced, posinglimits regarding its interoperability.Currently, developers commit to a single blockchain so-lution, and they cannot use the capabilities of more thanone blockchain (vendor lock-in). These isolated, incompatiblenetworks have resulted in silos of data and assets, which can-not be used from other networks. Blockchain interoperabilitysolutions are needed to enable asset and information transfer
This project has been supported by
The Linux Foundation as part ofthe
Hyperledger Summer Internships program under the
Towards BlockchainInteroperability with Hyperledger project. from one blockchain to another. However, interoperability forblockchains encounters challenges that make it different frominteroperability for other software networks.First, each interoperability solution should take into accountthe differences in the architecture of blockchain networks.Although all blockchains have an immutable ledger that storesthe history of assets, they usually reach a consensus on theorder of transactions using different algorithms. As a result,their underlying network and their validation mechanismscan be different from other blockchains. Each blockchainnetwork that participates in the interoperation is independentand in full control of their assets and information. Moreover,the interoperability solutions should not require significantchanges in the underlying blockchain networks, and it shouldbe usable with minimal effort for existing blockchains.This paper aims to tackle this problem by proposinga blockchain interoperability solution based on the pub-lish/subscribe architecture across permissioned blockchains.We have implemented a broker blockchain that acts as amiddleman in the interoperability process between the sourceand the destination networks. It is worth noting that since thebroker is itself a blockchain network, it is not a central author-ity and peers from the source and destination blockchains canalso participate in the governance of this network. The broker blockchain keeps a copy of the information that needs to beshared in the form of a topic . A topic has a name, message,publisher, and a set of subscribers. The publisher is the sourceblockchain network that wants to share the information. It isresponsible for creating the topic on the broker and publishingit to the corresponding topic whenever the information needsan update. The subscribers are the destination networks thatneed some information from the source network. As soonas the subscriber network subscribes to a topic, the brokernetwork notifies it whenever a change happens. This solutionenables interoperability between blockchains with minimaleffort.We used a performance benchmark tool to analyze the per-formance of the implemented prototype of the platform. Thethroughput and average latency for different functionalities ofthe broker network were investigated. The results indicate thatour network can handle hundreds of transactions per second.Moreover, the evaluations identified the
PublishToTopic func-tionality to be the broker network’s bottleneck. a r X i v : . [ c s . D C ] J a n he rest of this paper is organized as follows. Section IIgives a summary of the related work on blockchain inter-operability and blockchain-based publish/subscribe protocols.Section III introduces the system design details for the pro-posed interoperability solution. Section IV demonstrates theimplementation and deployment details of the platform, whilesection V presents its performance evaluation. Section VIoutlines some discussions on the design and evaluation of theplatform and section VII concludes the paper.II. R ELATED W ORK
In this section, we discuss the related work in the fieldof blockchain interoperability, as well as blockchain-basedpublish/subscribe protocols.
A. Blockchain Interoperability
The emergence of the blockchain interoperability researcharea has brought attention from both the industry andacademia. Blockchain interoperability projects have been tack-led as early as in 2016 [12].A recent survey classifies blockchain interoperability studiesin three categories: Cryptocurrency-directed interoperabilityapproaches, Blockchain Engines, and Blockchain Connec-tors [12]. Cryptocurrency-directed approaches are mostly in-dustry solutions that provide interoperability across publicblockchains. This category has a focus on asset interoperabilityand is divided into sidechains, hash lock time contracts, notaryschemes, and hybrid solutions. Sidechains allow for offloadingtransactions to a secondary chain, enhance performance, andprovide features that the main chain would not provide.Sidechains also enable the representation of a token from themainchain at the secondary chain. Some sidechain solutionsinclude the BTC Relay [13], Zendoo [14], and RSK [15].Hash lock time contract solutions enable cross-chain atomicoperations using smart contracts. Wanchain uses this schemeand provides loan services with cryptocurrencies [16]. Notaryschemes are centralized or decentralized entities that medi-ate token exchange (e.g., cryptocurrency exchanges). Finally,hybrid solutions combine characteristics from previous ap-proaches. The cryptocurrency-directed approaches only workfor transferring different types of cryptocurrencies betweenblockchain network. As a result, these approaches cannot beused for permissioned blockchains with arbitrary assets andsmart contracts, which are the focus of this paper.The second category is blockchain engines, which enablecreating customized blockchains that can interoperate by pro-viding reusable data, network, consensus, and contract layers.Platforms like Polkadot [17] and Cosmos [18] provide suchinfrastructure, with “free interoperability” among the instancesthey allow to create. These approaches are fundamentallydifferent from what has been proposed in this paper. In-stead of enabling blockchain interoperability for currentlyrunning blockchains, blockchain engines propose blockchainnetworks that are interoperable by design. As a result, thesesolutions cannot be used for currently running permissionedblockchains. The Blockchain Connector category is composed of inter-operability solutions that are not cryptocurrency-directed orblockchain engines. They include trusted relays, blockchainagnostic protocols, blockchain of blockchains solutions, andblockchain migrators. Each of these categories responds toa particular set of use cases. Trusted relays allow discov-ering the target blockchains, often appearing in a permis-sioned blockchain environment where trusted escrow partiesroute cross-blockchain transactions. An interesting trustedrelay approach is Hyperledger Cactus [19], the most recentHyperledger project aiming to connect a client to severalblockchains, whereby transactions are endorsed by trustedvalidators. Cactus focuses on providing multiple use casescenarios via a trusted consortium. Another trusted relay isproposed by Abebe et al. [20], which is an interoperabilitysolution between Fabric networks using Hyperledger Fabricchaincode and a protocol-buffer based communication pro-tocol. Blockchain agnostic protocols enable cross-blockchaincommunication between arbitrary distributed ledger tech-nologies, including refactoring and making changes to eachblockchain. Blockchain of blockchains are approaches thatallow users to build decentralized applications using multipleblockchains. Finally, blockchain migrators enable the state ofone blockchain to be migrated to another blockchain. Simpleblockchain migrations have already been performed, pavingthe direction for complex blockchain migrations [12].Our pub-sub solution is considered a trusted relay (al-though decentralized), as it contains a blockchain mediatingcommunication across heterogeneous blockchains [12]. Whiletrusted relays can provide a straightforward way of achievinginteroperability, most of them are not trustless (e.g., containa centralization point). Our solution is a decentralized trustedrelay that implements a pub/sub system, anchored on the trustthat underlying blockchains offer.
B. Blockchain-based Publish/Subscribe Protocol
The blockchain technology has been applied to the pub/subparadigm in a few previous studies. However, those studiesadopt blockchain to address the existing problems in otherareas, such as IoT [21], supply chain [22], multi-tenant edgecloud [23], and digital trading [24].Huang et al. [23] exploit blockchain technology to enhancethe security of pub/sub communications in multi-tenant edgeclouds. Most topic-based and broker-enabled pub/sub stream-ing systems use centralized cloud servers to store sensitivemetadata and access control lists (ACL), which can compro-mise the confidentiality, anonymity, and integrity of tenants’data. Alternatively, critical data such as ACL and identityinformation, as well as the hash of raw messages, and oper-ation logs, can be stored on the blockchain to guarantee datasecurity and integrity. Their smart contracts implement accesscontrol mechanisms to authorize requests from publishers andsubscribers.Trinity [22] proposes a blockchain-based distributed pub-lish/subscribe broker to solve the existing flaws of centralizedbrokers in IoT and supply chain monitoring applications.rinity has three main components: blockchain network, bro-ker, and clients. The blockchain network is responsible forconsensus in the system and persistent storage. The brokerhandles the communications between the blockchain networkand clients. The clients are the publishers and subscribersof the topics. The blockchain network is pluggable, and theauthors have used Tendermint, Hyperledger Fabric, Ethereum,and IOTA. The broker has used the Mosquitto MQTT broker,which provides a single point of failure.Zhao et al. [25] have proposed Secure Pub-Sub (SPS), whichprovides fair payment based on a reputation for publishers andsubscribers in cyber-physical systems. They use the Bitcoin’snetwork to enable payments between the entities, and theypropose a reputation mechanism that helps calculate the priceof data.Lv et al. [21] presents a decentralized privacy-preservingpub/sub model for IoT systems to solve centralized brokers’problems such as single point of failure, data tampering due tocorrupter brokers, and heavy encryption algorithms. The pre-sented model applies public-key encryption with equality test[26] and ElGamal [27] to protect participants’ (both publishersand subscribers) privacy. A system prototype is implementedand evaluated against the feasibility of the proposed model.Bu et al. [24] and Zupan et al. [28] have proposedblockchain-based pub/sub brokers to address the drawbacks oftraditional pub/sub systems such as privacy and accountability.However, they have not explained their implementation andevaluation in their studies.All these studies adopt blockchain to improve the central-ized predicaments in traditional pub/sub systems in distinctapplication domains, whereas our paper focuses on establish-ing robust and practical interoperability between permissionedblockchains with different architectures and infrastructures.To the best of our knowledge, our paper is the first studythat enhances blockchain interoperability utilizing the pub/subcommunication model across heterogeneous blockchains (Hy-perledger Fabric/Hyperledger Besu).III. S
YSTEM D ESIGN
In this section, we first discuss the design principles thata blockchain interoperability solution should follow. We thenpropose our interoperability solution and explain its compo-nents and message flow.
A. Design Principles
Blockchain interoperability aims to enable applications touse the assets and information available on blockchains otherthan their main blockchain network [12]. This allows for agreater range of applications. A blockchain interoperability so-lution should take into account the following design principles: • The blockchain networks are independent, and they mayhave different architectures. • The blockchain networks are in full control of their assetsand information. • The transfer protocol should be technology agnostic. • The interoperability solution should not require signifi-cant changes in the source and destination networks. • The blockchain networks should be able to incorporatethe solution with minimal effort.Following the mentioned principles, we present our solution,which allows interoperability based on a publish/subscribearchitecture.
B. Components
The platform proposed in this paper aims to solve theinteroperability problem of permissioned blockchains usingthe publish/subscribe pattern. When a blockchain wants to usethe data from another blockchain, there needs to be a way tofetch and transfer this data between the networks securely.Moreover, if the data changes in the source network, thedestination network should be notified of the change. Figure 1shows the architecture of the platform and the message flow.The publisher blockchain is the blockchain network thatsends the data, also referred to as the source network. For apublisher to participate in this platform, it needs to run theappropriate connector smart contract on its blockchain andenroll as a publisher in the broker. The publisher can thencreate as many topics as they want and use the connectorsmart contract to publish the changes to the topic.The subscriber blockchain is the blockchain network thatreceived the data, also referred to as the destination network.Similar to the publisher, the subscriber also needs to run theappropriate connector smart contract and enroll as a subscriber.It can then subscribe to any topic available on the brokerblockchain. Every time the topic changes, the broker notifiesthe subscriber by invoking the connector smart contract. Therecan exist many subscribers for a topic.The broker blockchain is the core component of the plat-form. It is a blockchain network that stores all the informationabout the topics and the blockchains that participate in theinteroperability process. Since the broker is a blockchain, op-erations regarding interoperation are immutable and traceable,providing a robust basis for accountability. The broker has twodifferent smart contracts, the connector smart contract andthe topics smart contract . The connector smart contract storesthe information about the participating blockchain networksand how the broker can interact with them. The topics smartcontract is responsible for storing the information about thetopics such as their publisher, subscribers, and the latestmessage.
C. Message Flow
The complete interoperation process in the platform isshown in Figure 1. For simplicity, only one publisher and onesubscriber blockchain are shown in this figure. However, foreach topic, there can be an unlimited number of subscribersand, in general, there is no limit on the number of publisherand subscriber blockchains. A detailed explanation of eachstep in Figure 1 follows:1) For any blockchain network to interact with the bro-ker blockchain, it must enroll in the connector smart
1) Enroll as a publisher
Broker BlockchainConnectorSmart Contract (2) Create a new topic(6) Publish to the created topic (3) Enroll as a subscriber(4) Subscribe to a topic
Publisher BlockchainConnectorApplication (5) Update topic
Subscriber BlockchainConnector Application (9) Topic is updated(8) Notify subscriber
Topics Smart Contract (7) Fetch subscribers
Fig. 1. Architecture of the platform and the message flow. contract. During this process, the information that thebroker needs to be able to interact with the blockchainis registered in the connector smart contract. As a result,the publisher is required to enroll in the connector smartcontract as a publisher. This step only needs to beperformed once, when the publisher wants to create itsfirst topic. It can then create topics or publish to existingones without the need to be enrolled again.2) A publisher that is registered in the connector smartcontract can create a new topic. In this step, the publisherneeds to specify a name for the topic and the initial topicmessage. This step only needs to be executed once foreach topic.3) Similar to the publisher blockchain, the subscriberblockchain should also enroll in the connector smartcontract. This step only needs to be done once, whenthe subscriber wants to subscribe to a topic for the firsttime.4) To receive a notification when a topic has been changed,the subscriber should subscribe to the topic by sendinga request to the topics smart contract. This results in thesubscriber being added to the list of all subscribers forthe topic. This step only needs to be performed once foreach topic.5) Whenever needed, the application in the publisherblockchain can update the topic by sending a requestto the connector smart contract.6) The connector smart contract sends a publish request tothe topics smart contract for the existing topic.7) As soon as a publish request is received by the topicssmart contract, the smart contract fetches the informationabout all the subscribers of the topic from the connectorsmart contract. It includes information on how the brokercan interact with each of the subscribers.8) The topics smart contract then uses the data fetched fromthe connector smart contract to notify all the subscribersof the change in the topic. This happens by sending anupdate request to the connector smart contract in eachof the subscriber networks. 9) In each subscriber network, the connector smart con-tract receives the update for the topic and notifies theapplication.IV. I
MPLEMENTATION AND D EPLOYMENT
A prototype of the proposed platform has been imple-mented as a proof-of-concept to demonstrate the feasibilityof the design. This project is a Hyperledger Labs open-source project , under the Apache License, Version 2.0. Toshow the interoperability capabilities of the platform, weimplemented two example subscriber networks, as well asan example publisher network. The broker and the examplepublisher are implemented using two different HyperledgerFabric V2 [29] networks. The two example subscribers areimplemented using Hyperledger Fabric V1.4 [30], [31], andHyperledger Besu [32]. In this section, the implementation anddeployment details of the broker and the example networks arediscussed. A. Broker Blockchain
The broker blockchain acts as a messaging broker betweenother blockchains to enable interoperability. When choosingthe blockchain solution to implement the broker network, wehad to ensure that the solution fits well with the needs ofthis platform. First, since we aim to address interoperabil-ity in permissioned blockchains, the broker also needs tobe permissioned. Moreover, many permissioned blockchainsare enterprise-level, and they may have privacy and gover-nance concerns, which our broker aims to addresses usingblockchain that enables immutability and traceability. We needa blockchain solution for the broker network that considersthese needs. Finally, the smart contracts that need to beimplemented on the broker blockchain are complicated, andthe blockchain needs to support this kind of smart contract.Hyperledger Fabric is an open-source permissionedblockchain that has been designed for enterprise use cases.Unlike the open permissionless blockchains that have scal-ability issues, Fabric enables high transaction throughput https://github.com/hyperledger-labs/pubsub-interop nd low transaction confirmation latency. The architectureof Hyperledger Fabric is highly modular and configurable,which enables customization for each specific use case. Manyblockchains only support smart contracts written in non-standard and domain-specific programming languages, makingit impossible to implement smart contracts that require aTuring-complete language. On the other hand, HyperledgerFabric supports smart contracts in general-purpose program-ming languages such as Go, Node.js, and Java [29].In the broker network, we leverage the capabilities ofHyperledger Fabric V2.2 to implement a messaging broker.We implement two chaincodes called the topics and theconnector to support the features needed for the broker. Thetopics chaincode is responsible for keeping all the topics andtheir corresponding details. In Hyperledger Fabric, everythingis stored as a key-value pair. In the topics smart contract,the key is a unique topic ID. The value is an object thatincludes the following properties: name, publisher, subscribers,and message. The name of a topic is a string value set bythe publisher when creating the topic. Each topic has onepublisher, the blockchain network that has registered the topicon the broker, which is the only blockchain that can makechanges to the topic. The subscribers’ property stores a list ofall the blockchains that have subscribed to the topic. It is worthmentioning that the publisher and the subscribers’ propertiesonly accept objects stored on the connector blockchain.The connector chaincode is responsible for storing theconnection details of other blockchain networks. Similar tothe topics chaincode, the key in the key-value pair used in thischaincode is a unique ID for each blockchain. The value is anobject that has the following properties: name, type, serverIP, port, extra information. The name is a string value thatcan be selected when enrolling in the network. Type showswhat kind of blockchain technology this network is using.Currently, support for Fabric and Besu has been implemented,and other blockchains will be added in future versions. Theserver IP and port are used by the broker blockchain to accessthe publisher or subscriber using an HTTP request. The extrainformation property stores network-specific details that maybe needed when interacting with the blockchains. For instance,for a Hyperledger Besu network, this includes the private key,address, and the contract application binary interface (ABI)that the broker should use to send a request to the Besunetwork. This kind of implementation allows our solutionto be extendable to other blockchains, independent of theirunderlying network.To better understand how the topics and connector chain-codes work, we need to discuss their implemented func-tionalities. Figure 2 shows the UML class diagram of theimplemented chaincodes. The Hyperledger Fabric contractAPI provides an interface for developing smart contracts andapplications. Each developed chaincode should extend thecontract class from this API and then implement the requiredlogic. In each smart contract, the InitLedger function is used tocreate a set of initial assets on the ledger when the chaincodeis deployed. In the topics chaincode, the
CreateTopic function is used to create a new asset of type topic. The
QueryTopic andthe
QueryAllTopics functions can be used to query one specifictopic and all the existing topics, respectively. The connectorchaincode implements the same initialize, create, and queryfunctionalities but for assets of type blockchain.
TopicsInitLedgerCreateTopicQueryTopicQueryAllTopicsSubscribeToTopicUnsubscribeFromTopicPublishToTopic ConnectorInitLedgerCreateBlockchainQueryBlockchainQueryAllBlockchains fabric-contract-api.Contract
Fig. 2. UML class diagram of the implemented chaincodes.
Other than the mentioned functions, the topics blockchainalso implements
SubscribeToTopic , UnsubscribeFromTopic ,and
PublishToTopic functionalities. When a destinationblockchain wants to get notified of a topic’s change, itsubscribes to that topic by invoking the
SubscribeToTopic function. The subscriber can also unsubscribe from the topicat any time by invoking the
UnsubscribeFromTopic function.Finally, the
PublishToTopic function is used by the sourceblockchain network when they want to update the topic’smessage. An invoke request to this function triggers updaterequests to all the subscribers of the topic. Algorithm 1 showsthe detailed implementation of the
PublishToTopic method.First, the broker retrieves the latest version of the topic fromthe ledger. In the case that no topic was found, it immediatelythrows an error. Next, the topic’s message is updated with thenew message value and the topic’s state is put to the ledger.The next step is for the broker to notify all the subscribers.For each of the subscribers of the topic, the blockchain objectis queried from the connector contract. This inter-chaincodecommunication is also shown in Figure 2. Then, given thetype of subscriber blockchain, the steps to invoke the remotenetwork are followed.
B. Subscriber Blockchains
The subscriber, or destination blockchain, is the blockchainthat requires information from another blockchain to run atask. For the subscriber to be able to participate in the platform,it needs to have the appropriate connector smart contractdeployed on it. We have implemented subscriber connectorcontracts for Hyperledger Fabric V1.4 and Hyperledger Besu.However, the connector is a simple smart contract that canalso be developed by the owners of the subscriber blockchain.This smart contract needs to keep track of the topics that thesubscriber has subscribed to and store their latest version for lgorithm 1:
PublishToTopic Method
Input: topicID, newMessage
Result:
Subscribers are notified of the new message topicState ← getState(topicID) if !topicState then throw error end topicState.message ← newMessage putState(topicID, topicState) for subID in topicState.subscribers do subState ← query subID from connector contract if subState.type = Fabric then follow steps to invoke a remote Fabric network else if subState.type = Besu then follow steps to invoke a remote Besu network end end other smart contracts to access at any time. Two examplesubscriber networks have been implemented to demonstratethe interoperability capabilities of the platform.The first example subscriber is implemented using Hyper-ledger Fabric V1.4. The second example subscriber is imple-mented using Hyperledger Besu, an open-source Ethereumclient that supports private and permissioned blockchains.Besu can create networks that work based on a proof of work(PoW) or a proof of authority (PoA) consensus algorithm. Inthis work, we implemented a PoW network using Besu, whichcan be thought of as a private Ethereum network. We thenimplemented a connector smart contract in Solidity to keep arecord of the subscribed topics. C. Publisher Blockchains
The publisher, or the source blockchain, is the blockchainnetwork that needs to send information to other blockchains.Similar to what we have in the subscriber blockchain, aconnector smart contract is also required for the publishers.However, the connector is slightly different in the publisher.The publisher connector should not only keep track of thetopics, but it should also connect to the broker blockchainto publish the topics. We implemented an example publishernetwork using Hyperledger Fabric V2.2.V. E
XPERIMENTAL E VALUATION
In this section, we focus on evaluating the performance ofthe implemented prototype of the broker blockchain. The goalis to see how the throughput and latency of the system changesin different scenarios. We have conducted two series of exper-iments to achieve this goal. The first set of experiments aimsto show the performance metrics of different functionalities inthe broker blockchain. The second set of experiments focuseson the publish function, which is the most important and time-consuming component of the broker blockchain.We have used Hyperledger Caliper [33] to run the ex-periments. Hyperledger Caliper is an open-source blockchainperformance benchmark tool that allows performance mea-surement for different blockchains, such as Hyperledger Fab-ric, Ethereum, Hyperledger Besu. In Hyperledger Caliper, the workloads or benchmarks are responsible for generating thecontent of each transaction that is sent to the blockchainnetwork. Given the network and benchmark configurations,Caliper uses a set of independent workers to send sched-uled requests to the blockchain network and monitor theresponse. When the tests are finished, Caliper generates aperformance report consisting of the average throughput andminimum, maximum, and average latency throughout the test.The throughput shows the number of transactions that wereprocessed in the system in a given time. The latency showsthe amount of time it takes for a transaction to be finished andadded to the ledger.Table I summarizes the specifications of each componentin the experimental evaluation. We have set up HyperledgerCaliper on a separate machine to ensure that its process doesnot affect the performance of the broker network. We usefive workers, a fixed rate controller, and a test duration of60 seconds for each benchmark round. The broker networkis implemented using Hyperledger Fabric V2.2 with two peerorganizations and an orderer organization, each with an in-dependent certificate authority. Each of the peer organizationshosts one peer node, and the orderer uses Raft implementation.Two chaincodes have been implemented that run on onechannel. The Fabric subscriber, implemented using FabricV1.4, has two organizations, each hosting two peers. Thewhole subscriber network uses one Solo orderer and oneFabric certificate authority. The Besu subscriber implementsa private Ethereum network with PoW consensus algorithm.The publisher has been implemented using Hyperledger FabricV2.2 with the same configurations as the broker network.
TABLE IE
XPERIMENTAL EVALUATION SETUP
Component Type CPU RAM DiskCaliper Benchmark N/A 8 vCPU 30 GB 288 GBBroker Fabric V2.2 8 vCPU 30 GB 288 GBFabric Subscriber Fabric V1.4 2 vCPU 7.5 GB 36 GBBesu Subscriber Besu 2 vCPU 7.5 GB 36 GBPublisher Fabric V2.2 2 vCPU 7.5 GB 36 GB
The first set of experiments focuses on the performanceevaluation of broker blockchain. In these experiments, weconduct a series of tests using Hyperledger Caliper for eachfunctionality that broker blockchain offers. Figure 2 summa-rizes all these functionalities. Each type of transaction goesthrough a specific set of steps in Hyperledger Fabric, whichhighly influences the response time for that transaction. Forinstance, an invoke transaction goes through endorse, orderand commit steps. On the other hand, a query transaction isnot transferred to the orderer, and the response is immediatelysent back by the peer. The create actions in the connector andtopics smart contract are invoke actions that have very similarimplementations. The same goes for the query actions in thetwo smart contracts. As a result, it would be repetitive to run S e n d R a t e ( t p s ) PublishSubUnsubQueryCreate0100200300400 T h r o u g h p u t ( t p s ) PublishSubUnsubQueryCreate0 5 10 15 20 25 30Time (min)020406080 A v g . L a t e n c y ( s ) PublishSubUnsubQueryCreate
Fig. 3. The trend of system throughput and average latency for vari-ous functionalities throughout time with the change of request send rate.The words publish, sub, unsub, query, and create in the plots stand forPublishToTopic, SubscribeToTopic, UnsubscribeFromTopic, QueryTopic, andCreateTopic functions, respectively. performance evaluation experiments for both smart contracts.Therefore, we run the experiments on the topics smart contract.The topics smart contract has five important functionalities:create a topic, query a topic, publish to a topic, subscribe to atopic, and unsubscribe from a topic. For each of these actions,we run a set of experiments by changing the transaction sendrate in the Hyperledger Caliper benchmark. The goal is tosee how the system’s throughput and average latency changeswhen the send rate is changed. Figure 3 shows the details ofthese experiments. It can be seen that the send rate followsthe same pattern for all the actions except for
PublishToTopic .The reason for this difference is that the
PublishToTopic actiontakes more time and needs more resources to run comparedto other actions. Consequently, broker blockchain’s hardwarelimits are reached when the network receives more thanroughly 100 publish transactions in each second. We discussthe behaviour of the network with different
PublishToTopic requests in the second set of experiments shown in Figure 4.As a result of this limitation, we lowered the send rate for the
PublishToTopic action in our experiments.It can be seen in Figure 3 that the
SubscribeToTopic , Un- subscribeFromTopic , and
CreateTopic have similar behavioursunder the same send rate. These three actions are of typeinvoke. Since an invoke transaction proposes a change in theblockchain, it needs to go through the consensus algorithm,which can be time-consuming. Since the three actions areof the same type, and none need heavy computations inexecution, the system throughput and latency for all of themare similar. As shown in the experimentation results, when thesend rate is lower than a threshold (around 160 TPS in thiscase), the throughput is the same as the send rate, and theaverage latency is only a few hundred milliseconds (around100 to 300 milliseconds). This shows that with send ratesbelow the threshold, all transactions are processed immedi-ately. When the number of create, subscribe, or unsubscribetransactions sent in each second is more than the threshold, thebroker network’s processing limit is reached. The throughputis limited to the broker’s maximum capacity (around 160 TPS),and the transactions are queued before being processed, whichresults in an increase in the latency. Figure 3 shows thatwhen the send rate for the create, subscribe, or unsubscribetransactions is around 210 TPS, the average latency increasesto about 11 seconds. The latency keeps increasing with highersend rates and reaches approximately 50 seconds with a sendrate of 360 TPS.The
QueryTopic action is different from the previous ones.Since a query transaction does not go through the consensusprotocol, its process is much faster. The send rate pattern usedfor the query is similar to that of create, subscribe, and unsub-scribe. However, the throughput and average latency act verydifferently. The throughput follows the same pattern as thesend rate, and the average latency is around 10 millisecondsthroughout the whole experiment. These results show that thisexperiment does not reach the process limit for
QueryTopic .Finally, the
PublishToTopic is similar to create, subscribe,and unsubscribe because they are all invoke transactions.However, the publish action requires stronger computations.As mentioned earlier, since the publish action needs moretime and computational resources, we use a different sendrate pattern. If we were to use the same send rate, the brokerblockchain’s hardware limits would be reached, resulting in theexperiments being halted. We discuss this in more detail in thesecond set of experiments shown in Figure 4. To ensure thatthe performance of the remote source and destination networksdo not influence the performance evaluation of the brokernetwork, we only send dummy requests to the subscribernetworks during the experiments. It can be observed fromFigure 3 that the publish action reaches the processing limitof the broker network much faster than the other invoketransactions. With send rates of about 70 TPS and more, thethroughput is limited to 65 TPS. The average latency for thepublish action has more fluctuations compared to other invokeactions. The main reason for this fluctuation is that in thepublish method, depending on the number of subscribers thatthe topic has, the processing time can vary. In this experiment,the average latency gets as high as 80 seconds, with the sendrate of 110 TPS. R a t e ( t p s ) Send Rate (tps)Throughput (tps)0 2 4 6 8 10Time (min) A v g . L a t e n c y ( s ) S u cc e ss R a t e ( % ) Fig. 4. The trend of system throughput, average latency, and request successrate throughout time with the change of send rate.
Given the limits of the
PublishToTopic action, we decidedto run some additional experiments on this type of transaction.This experiment aims to find the broker network’s limits anddiscover what happens when the limit is reached. In theprevious experiment, we discovered that the processing limitfor the publish transactions is reached at the sent rate of around70 TPS. We also observed that the latency increases and thethroughput is limited for send rates above 70 TPS and below110 TPS. However, we would like to know what happensif the send rate is more than 110 TPS. In this experiment,we linearly increase the send rate from 50 to 150 TPS andobserve the throughput, latency, and transaction success rate.Figure 4 shows the results of this experiment. Similar to theprevious experiment, we see that the throughput is limited, andthe latency is increased when the send rate reaches 70 TPS.Nevertheless, the interesting change happens at the 120 TPSsend rate. At this point, a significant drop in the throughputand a significant rise of latency are observed. Moreover, thetransaction success rate is not 100% anymore. From this pointon, a portion of the transactions fail since the broker networkhas reached its hardware limits.VI. D
ISCUSSION AND F UTURE W ORK
To enable blockchain interoperability, we have proposedthe use of a broker blockchain as a middleman. The brokerblockchain acts as a decentralized trusted relay between thesource and destination network. Using a relay enables theinteroperating networks to transfer data with minimal effort.Verifying the data and handling the communications betweendifferent blockchain networks can be delegated to the relay. Asa result, there is no need for source and destination networksto make fundamental changes to their underlying structure.The relay network is also a blockchain network; while ex-ploiting all desirable features offered by blockchain, it runssmart contracts implementing the interoperability functional- ity. Therefore, the broker blockchain allows the interoperationto be seamless, transparent, and secure.The platform proposed in this paper stores the destinationand source blockchains as assets on the distributed ledger.As a result, a large number of blockchains can be supportedas there are no limits on the number of assets. A study onthe performance of Hyperledger Fabric V1.1 shows that thenetwork can scale to 100+ peers [34], [35]. As the brokernetwork has been implemented using Fabric V2.2, we expectthis number to be even higher in our network. Therefore, atleast 100 peers can participate in the governance of the brokerblockchain.In the current prototype of our platform, every participantcan subscribe to all existing topics, and there is no accesscontrol mechanism implemented. The private data featurepresented by Hyperledger Fabric can be utilized to establishprivate channels in the broker blockchain and keep data topicsseparate from other organizations. Furthermore, an access con-trol mechanism can be added to our pub/sub system to controlthe flow of data at a more granular level, for instance, thedecentralized access control proposed by Rouhani et al. [36].It is also possible to conduct authorization processes withminimal information disclosure if one uses the novel self-sovereign based access control model [37]. This way, wecould model each blockchain as an agent to prove that theyown specific credentials needed for the access control process.Moreover, the publisher network may choose only to make atopic available to a subset of subscribers. Access control canbe used to manage the blockchains that can access each topic.VII. C
ONCLUSION
With blockchain technology gaining popularity in academiaand industry, many blockchain networks are being intro-duced worldwide. These networks are highly isolated andincompatible with each other, resulting in silos of data andassets. Blockchain interoperability solutions can revolutionizethis technology by enabling data and asset transfers betweenhomogeneous and heterogeneous blockchains. In this paper,we proposed a blockchain interoperability solution based onthe publish/subscribe architecture. Our solution consists of abroker blockchain that keeps a record of the data being trans-ferred between blockchain networks. The blockchains that aimto participate in the interoperability can connect to the brokernetwork as publishers or subscribers, depending on their role.A prototype of the broker blockchain has been implementedusing Hyperledger Fabric. Moreover, an example publisherand two example subscribers have been implemented usingHyperledger Besu and two versions of Hyperledger Fabric toshow that the design works for heterogeneous blockchains.The network’s performance has been analyzed using a bench-mark tool to identify the platform’s limits and bottlenecks.The implementation and evaluations indicate the feasibility ofthe idea with satisfactory performance, and the bottleneck isidentified to be the process of publishing a new message toa topic. Finally, a discussion on the extensibility, scalability,and possible improvements of the system is presented.
EFERENCES[1] S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system.” https://bitcoin.org/bitcoin.pdf, 2008. Last accessed 2020-07-17.[2] G. Wood et al. , “Ethereum: A secure decentralised generalised transac-tion ledger,”
Ethereum project yellow paper , vol. 151, no. 2014, pp. 1–32, 2014.[3] B. Chase and E. MacBrough, “Analysis of the xrp ledger consensusprotocol,” arXiv preprint arXiv:1802.07242 , 2018.[4] T.-T. Kuo, H.-E. Kim, and L. Ohno-Machado, “Blockchain distributedledger technologies for biomedical and health care applications,”
Journalof the American Medical Informatics Association , vol. 24, no. 6,pp. 1211–1220, 2017.[5] S. Rouhani, L. Butterworth, A. D. Simmons, D. G. Humphery, andR. Deters, “Medichain tm: a secure decentralized medical data assetmanagement system,” in , pp. 1533–1538, IEEE,2018.[6] T. M. Fern´andez-Caram´es and P. Fraga-Lamas, “A review on the use ofblockchain for the internet of things,”
IEEE Access , vol. 6, pp. 32979–33001, 2018.[7] C. Fan, H. Khazaei, Y. Chen, and P. Musilek, “Towards a scalable dag-based distributed ledger for smart communities,” in , pp. 177–182, IEEE, 2019.[8] R. Belchior, M. Correia, and A. Vasconcelos, “JusticeChain: UsingBlockchain To Protect Justice Logs,” in
CoopIS 2019: 27th InternationalConference on Cooperative Information Systems , 2019.[9] R. Belchior, A. Vasconcelos, and M. Correia, “Towards Secure, Decen-tralized, and Automatic Audits with Blockchain,” in
European Confer-ence on Information Systems , 2020.[10] S. Ghaemi, H. Khazaei, and P. Musilek, “Chainfaas: An openblockchain-based serverless platform,”
IEEE Access , vol. 8, pp. 131760–131778, 2020.[11] J. H. Park and J. H. Park, “Blockchain security in cloud computing: Usecases, challenges, and solutions,”
Symmetry , vol. 9, no. 8, p. 164, 2017.[12] R. Belchior, A. Vasconcelos, S. Guerreiro, and M. Correia, “A surveyon blockchain interoperability: Past, present, and future trends,” arXivpreprint arXiv:2005.14282 , 2020.[13] Ethereum Foundation and Consensys, “BTC-relay: Ethereum contractfor Bitcoin SPV,” 2015.[14] A. Garoffolo, D. Kaidalov, and R. Oliynykov, “Zendoo: a zk-SNARKVerifiable Cross-Chain Transfer Protocol Enabling Decoupled and De-centralized Sidechains,” tech. rep., 2020.[15] S. Lerner tech. rep., RSK, 2015.[16] J. Lu, B. Yang, Z. Liang, Y. Zhang, S. Demmon, E. Swartz, and L. Lu,2017.[17] G. Wood, “Polkadot: Vision for a Heterogeneous Multi-Chain Frame-work,”
Whitepaper , 2017.[18] J. Kwon and E. Buchman, “Cosmos Whitepaper,” tech. rep., 2016.[19] H. Montgomery, H. Borne-Pons, J. Hamilton, M. Bowman, P. Somogy-vari, S. Fujimoto, T. Takeuchi, T. Kuhrt, and R. Belchior, “HyperledgerCactus Whitepaper.” https://github.com/hyperledger/cactus/blob/master/whitepaper/whitepaper.md, 2020. Last accessed 2020-09-28.[20] E. Abebe, D. Behl, C. Govindarajan, Y. Hu, D. Karunamoorthy,P. Novotny, V. Pandit, V. Ramakrishna, and C. Vecchiola, “Enablingenterprise blockchain interoperability with trusted data transfer (industrytrack),” in
Proceedings of the 20th International Middleware ConferenceIndustrial Track , pp. 29–35, 2019.[21] P. Lv, L. Wang, H. Zhu, W. Deng, and L. Gu, “An iot-oriented privacy-preserving publish/subscribe model over blockchains,”
IEEE Access ,vol. 7, pp. 41309–41314, 2019.[22] G. S. Ramachandran, K.-L. Wright, L. Zheng, P. Navaney, M. Naveed,B. Krishnamachari, and J. Dhaliwal, “Trinity: A byzantine fault-tolerantdistributed publish-subscribe system with immutable blockchain-basedpersistence,” in , pp. 227–235, IEEE, 2019.[23] B. Huang, R. Zhang, Z. Lu, Y. Zhang, J. Wu, L. Zhan, and P. C.Hung, “Bps: A reliable and efficient pub/sub communication model withblockchain-enhanced paradigm in multi-tenant edge cloud,”
Journal ofParallel and Distributed Computing , 2020. [24] G. Bu, T. S. L. Nguyen, M. P. Butucaru, and K. L. Thai, “Hyperpub-sub: Blockchain based publish/subscribe,” in , pp. 366–3662, IEEE, 2019.[25] Y. Zhao, Y. Li, Q. Mu, B. Yang, and Y. Yu, “Secure pub-sub: Blockchain-based fair payment with reputation for reliable cyber physical systems,”
IEEE Access , vol. 6, pp. 12295–12303, 2018.[26] G. Yang, C. H. Tan, Q. Huang, and D. S. Wong, “Probabilistic publickey encryption with equality test,” in
Cryptographers’ Track at the RSAConference , pp. 119–131, Springer, 2010.[27] T. ElGamal, “A public key cryptosystem and a signature schemebased on discrete logarithms,”
IEEE transactions on information theory ,vol. 31, no. 4, pp. 469–472, 1985.[28] N. Zupan, K. Zhang, and H.-A. Jacobsen, “Hyperpubsub: a decen-tralized, permissioned, publish/subscribe service using blockchains,” in
Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference:Posters and Demos , pp. 15–16, 2017.[29] Hyperledger, “Hyperledger fabric v2.2 documentation.”https://hyperledger-fabric.readthedocs.io/en/release-2.2/, 2020. Lastaccessed 2020-10-22.[30] Hyperledger, “Hyperledger fabric v1.4 documentation.”https://hyperledger-fabric.readthedocs.io/en/release-1.4/, 2019. Lastaccessed 2020-10-22.[31] E. Androulaki, A. Barger, V. Bortnikov, C. Cachin, K. Christidis,A. De Caro, D. Enyeart, C. Ferris, G. Laventman, Y. Manevich, et al. ,“Hyperledger fabric: a distributed operating system for permissionedblockchains,” in
Proceedings of the thirteenth EuroSys conference
IEEE Access , vol. 8,pp. 126927–126950, 2020.[35] E. Androulaki, A. Barger, V. Bortnikov, C. Cachin, K. Christidis,A. De Caro, D. Enyeart, C. Ferris, G. Laventman, Y. Manevich,S. Muralidharan, C. Murthy, B. Nguyen, M. Sethi, G. Singh, K. Smith,A. Sorniotti, C. Stathakopoulou, M. Vukoli´c, S. W. Cocco, and J. Yellick,“Hyperledger fabric: A distributed operating system for permissionedblockchains,” in
Proceedings of the Thirteenth EuroSys Conference ,EuroSys ’18, (New York, NY, USA), Association for Computing Ma-chinery, 2018.[36] S. Rouhani, R. Belchior, R. S. Cruz, and R. Deters, “Dis-tributed Attribute-Based Access Control System Using a PermissionedBlockchain,” arXiv preprint , 2020.[37] R. Belchior, B. Putz, G. Pernul, M. Correia, A. Vasconcelos, andS. Guerreiro, “SSIBAC : Self-Sovereign Identity Based Access Con-trol,” in