Latency Modeling of Hyperledger Fabric for Blockchain-enabled IoT Networks
Sungho Lee, Minsu Kim, Jemin Lee, Min-Soo Kim, Ruei-Hau Hsu, Tony Q. S. Quek
11 Latency Modeling of Hyperledger Fabric forBlockchain-based IoT (BC-IoT) Networks
Sungho Lee, Minsu Kim, Jemin Lee,
Member, IEEE ,Ruei-Hau Hsu,
Member, IEEE , and Tony Q. S. Quek,
Fellow, IEEE
Abstract —With the worldwide growth of IoT industry, theneed for a strong security level for IoT networks has alsoincreased, leading to blockchain-based IoT (BC-IoT) networks.While blockchain technology is leveraged to ensure data integrityin a distributed manner, Hyperledger Fabric (HLF) attractsattention with its distinctive strong point without requiringthe power-consuming consensus protocol, that is, proof-of-work(PoW). However, even though such security concerns can bemitigated using HLF, the additional processing time spent in HLFmay emerge as another issue because most IoT devices handlereal-time and latency critical jobs. This problem still remainsunresolved because of the absence of a HLF latency model and aparameter setup guideline to reducing the mean latency. In thispaper, therefore, we develop a HLF latency model for HLF-basedIoT networks based on probability distribution fitting, by whichmean latency prediction is facilitated once probable configurationenvironments are determined, in terms of the block size, block-generation timeout, and transaction generation rate parameters.Furthermore, we conclude by analyzing the impacts of influentialHLF parameters on the mean latency, in order to provide insightsnot only on optimizing the mean latency, but also on coping withlong mean latency.
I. I
NTRODUCTION
Blockchain technology has been widely studied since thefirst paper on Bitcoin was published to propose a new type ofelectronic monetary system. The paper points out the singlepoint of failure (SPOF) problem stemming from the currentcentralized cash system with a central authority, and provides asolution building a distributed monetary system to the SPOFproblem. Based on the underlying structure of the solution,which is blockchain, a whole variety of blockchain-baseddistributed platforms have emerged with different purposesfrom one another, accompanying a wide range of blockchain-related applications and research topics. As one of the exam-ples, a blockchain-based IoT (BC-IoT) network is introducedto fulfill new security requirements of IoT environments.Considering blockchain technology can not only ensure dataintegrity, but also ameliorate security concerns in a distributedmanner, various blockchain platforms have been explored anddiscussed for IoT devices with limited resources [1].
Corresponding author is J. Lee.S. Lee, M. Kim, and J. Lee are with Daegu Gyeongbuk Insti-tute of Science and Technology (DGIST), 333, Techno Jungang-daero,Daegu, Republic of Korea 42988 (e-mail: { seuho2003, kms0603,jmnlee } @dgist.ac.kr ).R. -H. Hsu is with National Sun Yat-sen University, 70 Lienhai Rd., Kaohsi-ung 80424, Taiwan, R.O.C. (e-mail: [email protected] ).T. Q. S. Quek is with Information Systems Technology and Design Pillar,Singapore University of Technology and Design, Singapore 487372 (e-mail: [email protected] ). The BC-IoT network refers to a particular IoT networkwhose underlying structure is on the basis of blockchaintechnology for IoT device data management. For a blockchainsystem, it is imperative to select a consensus protocol fordata integrity. Although proof-of-work (PoW) is the mostpopular and general choice, it is too power-consuming tobe unsuitable for power-constrained IoT devices because itis mandatory for miners to continually solve hash puzzlesfor data integrity. In this sense, Hyperledger Fabric (HLF)deserves consideration as HLF-based IoT networks becauseit supports a couple of alternative consensus methods withoutburdening IoT devices with high power consumption. Besides,its strict membership rule not only prevents privacy leakage,but also enables excluding unauthorized access, prohibittingattackers from working as legitimate members in HLF-basedIoT networks.However, from the viewpoint of IoT devices that handlereal-time jobs, it is also important to take the HLF latencyinto account on top of security concerns. Otherwise, theymay fail to accomplish latency critical tasks on time. In thissense, it is desirable to predict and reduce the mean latencyin order to prevent time sensitive jobs from spending toolong time in HLF. However, the absence of a HLF latencymodel leaves this problem unsettled. To be specific, HLF hasa sophisticated structure, in which several different nodes andcomponents simultaneously affect the performance, which willbe explained in detail in Section III. It signifies that there existmany considerations for the optimal parameter setup to achievethe lowest latency. In addition, the structural complexitycauses difficulty in constructing a testbed blockchain network,complicating performance evaluation of every use case andscenario. Thus, it is almost impossible to fulfill the lowestlatency without a HLF latency model.In this paper, we establish a latency model of HLF for aspecific BC-IoT network (i.e., a HLF-based IoT network), inwhich latency critical tasks are generally handled. We alsofocus on two important blockchain parameters (i.e., block sizeand block-generation timeout), and analyze their impacts onthe mean latency to provide guidelines to reducing it. Thecontributions of this paper are listed in detail as follows: • We leverage HLF v1.3 other than obsolete one (i.e., HLFv0.6) for latency modeling. Providing a latency modelbased on probability distribution fitting, we exhibit thatthe total latency of HLF is modeled with the gammadistribution within a certain configuration range. Thiscontributes not only to forecasting the mean latency ofHLF-based IoT networks in practice, but also to preparing a r X i v : . [ c s . D C ] F e b the optimal setup for the lowest latency. • Focusing on the block size and block-generation timeout,we analyze their impacts on the mean latency with vary-ing their configuration values. This analysis contributesto comprehending the methods for decreasing the meanlatency from each parameter’s perspective.The remainder of this paper is organized as follows: SectionII discusses related work and limitations. Section III providesa brief overview of HLF based on the internal transactionflow and essential components, instantiating HLF-based IoTnetworks. Section IV explains the methodology for probabilitydistribution fitting and proposes our latency model. Section Vdiscusses the impacts of two influential HLF parameters onthe mean ledger-commitment latency. Lastly, the conclusionsof this paper are organized in Section VI.II. R
ELATED W ORK
In this section, we introduce previous work that evaluatesthe performance of HLF in various aspects (e.g., throughput,latency, scalability, etc.), and that provides a performancemodel. Especially, the former studies are usually focused onassessing and exploring performance trends with various HLFsetups as well as in different environments from each other[2]–[8].Some studies are based on the obsolete version, which isHLF v0.6. The performance of HLF v0.6 is compared tothat of other private blockchain platforms (i.e., Ethereum andParity) in [2] and [3]. These papers explore their performancedifferences among the three private blockchain platforms,using a newly developed framework that they propose tobenchmark private blockchains. Another comparative studybetween Ethereum and HLF v0.6 is conducted in [4]. In [4],their performance differences in the execution time, averagelatency, and throughput are explored with varying numberof transactions, especially when the blockchain networks arecongested due to many transactions.Since HLF v1.0, it has a different structure from theobsolete version. Therefore, many researchers lately focus onthe new version when evaluating the performance of HLF.An empirical study is conducted to understand the overallperformance of HLF with varying blockchain parameter values[5]. In [5], experiments are performed to reveal how HLFperformance trends are changed with controllable parameters.Besides, simple optimizations for improving the overall systemperformance are also performed based on the action items thatthey list. With emphasis on the block size and batch-timeout(i.e., block-generation timeout), the effects of the two param-eters on the average throughput and latency are empiricallydemonstrated in [6]. In particular, this paper considers thatan increase in number of endorsing peers may also affectthe performance. Another performance evaluation study isconducted in [7]. This paper considers three parameters andperform experiments for each case to evaluate HLF againstthree performance metrics: throughput, latency, and scalability.Note that specific parameters considered in [7] are transactionarrival rates, number of transactions, and number of simul-taneous transactions. The performance differences between the old version and the new version of HLF are explored in[8]. Specifically, with the various number of total generatedtransactions, this paper mainly shows how an invoke functionand query function affect the execution time, average latency,and throughput in the different two versions of HLF.Particularly, some studies explore the performance fromthe perspective of specific cases in [9] and [10]. As oneof the specific use case studies, a HLF network dedicatedto IoT applications is discussed to evaluate and improve itsperformance [9]. In [9], a new algorithm using multi-batchedscheduling method, which is called as
Accelerator , is proposedand leveraged to assess the established HLF network forIoT devices against two performance metrics: throughput andlatency. Data updates in HLF occurs in all organizations, wherethe data are retained, and the next block to update the ledgerhas to await until the previous one is completely committedin all the participating organizations. In this sense, the impacton the performance is observed when block propagation isintentionally impeded in [10].The HLF components-related performance is evaluated in[11]–[13]. HLF supports several different programming lan-guages to construct a chaincode, which is also referred toas a smart contract. In [11], performance differences amongthe available languages are explored for a query and invokerequest. Practical Byzantine fault tolerance (PBFT) is to enablea distributed system to endure malicious behavior and nodefailures. This protocol is provided as a consensus protocolin HLF v0.6. In [12], performance evaluation of a PBFT-based ordering service is performed with designing maliciousbehavior cases, and this shows that malicious behavior is alsoimpactful upon the performance. Every peer in HLF possessesa database to store data. During the data update process, eachtransaction in a block updates the database. With regard tothis process, the database system performance is characterizedin [13]. In addition, performance improvement is fulfilledby GoLevelDB optimization work. However, even thoughthe aforementioned research [2]–[13] helps to understand thecomprehensive performance trends of HLF, it cannot be usedto predict the performance of a HLF network with a givenparameter setup.System modeling studies have been performed to designperformance models of HLF [14]–[17]. The PBFT consen-sus process, which is the consensus protocol in HLF v0.6,is modeled with empirically obtained data using stochasticreward nets (SRNs) in [14]. Based on the proposed SRNmodel, the scalability of PBFT is estimated by measuring themean time to finish the consensus process with large numberof nodes. In [15], furthermore, for the entire transaction-processing stages of HLF, a performance model of HLF witha Kafka/ZooKeeper-based ordering service is designed usingSRNs. The proposed SRN model of HLF is on the basis oftest runs, which are performed to decide appropriate parametervalues inside the SRN model. An analytical model of a HLF-based system is proposed in [16] using generalized stochasticpetri nets (GSPN). Furthermore, this paper also identifiesthe performance bottleneck based on the GSPN model andproposes a mathematical configuration selection approach toachieve the best throughput. In order to analyze the transaction process performance of HLF, a hierarchical model of HLF isdeveloped in [17]. The model accuracy is assessed throughnumerical analysis and simulations. The aforementioned HLFmodeling studies have at least one of the limitations in thefollowing: PBFT does not currently available since HLF v1.0,but works with the obsolete version only (i.e., HLF v0.6).Although a minor version of HLF v1.3 tailored for PBFThas been implemented, it is not an official release [18].Therefore, it is difficult to consider that performance modelsof HLF v0.6 with PBFT are practical for now. Besides, thedistributions of HLF latency types have not been exploreduntil the present, leading to difficulty in analyzing the HLFsystem in a stochastic manner.III. H
YPERLEDGER F ABRIC O VERVIEW
The Hyperledger Project aims to promote blockchain tech-nology by fostering open-source blockchain frameworks, vari-ous blockchain tools, and blockchain libraries [19]. In contrastto the previous blockchain platforms, where deterministicprogramming languages are compulsory to prohibit ledgersfrom diverging, HLF, which is one of the frameworks of thisproject, is to develop a private modular blockchain platformnot only enabling several business logics to be written ingeneral-purpose programming languages, but also preventingblockchain forks [20]. In order to support non-deterministicprogramming languages and to avoid blockchain forks simul-taneously, HLF adopts a novel structure, in which transactionsgo through three phases. In this section, we first provide thedefinitions of some significant system components and param-eters in HLF, then demonstrate the three-phased transactionflow. Note that we refer to the Hyperledger official websiteand Read the Docs [19] [21] to define the system componentswith accuracy in this paper.
A. System Components1) Peer:
The peer refers to a node which serves as amember in a HLF network. In HLF, the peer is mainlyclassified under three types: committing peer (i.e., committer),endorsing peer (i.e., endorser), and ordering peer (i.e., orderer). • The committing peer refers to a normal peer allowed tomaintain a copied ledger and to validate newly deliveredblocks. Note that committing peers are not empoweredto play the unique roles of endorsing and ordering peers(e.g., transaction proposal simulation, transaction order-ing, and block generation). • The endorsing peer refers to a particular committing peerentitled to hold and to access smart contracts, with whichtransaction proposals from clients are simulated againsttheir copied ledgers. In other words, endorsing peers arereferred to as particular peers qualified for generatingeach of their simulation results of transaction proposalsbased on the copied ledgers, using smart contracts. • The ordering peer refers to a node consisting of theordering service that is a node cluster taking charge ofarranging transactions in chronological order and produc-ing a new block. Ordering peers perform the pre-definedconsensus protocol on the new block, then the block isconveyed to peers via the gossip protocol.
2) Channel:
The channel is a sub-network, in which aset of peers are gathered together for data confidentiality.Every channel is registered by the ordering service in thechannel configuration block generated by the blockchain ser-vice provider, therefore, a peer is required to submit thechannel configuration block when joining a channel as anew member. The peers in the same channel may belong todifferent organizations from one another, however, they shareand maintain the identical ledger without allowing the otherpeers to tamper the ledger. In this paper, we call this channelcomponent as a HLF channel in order to avoid confusion.
3) Ledger:
The ledger is composed of a blockchain and thestate database. The blockchain includes all transactions, whichhave been generated until the present as a immutable chain ofblocks, while the state database contains the current valuesof key-value pairs, which have been modified and created byclients [21]. Each HLF channel has its own channel-specificledger, and peers can participate in multiple HLF channels.Thus, a peer holds and maintains each copied ledger of theHLF channels, in which it exists as a member.
4) Organization:
The organization refers to a group, whichpeers appertain to. An organization is joined to a HLF networkby the blockchain service provider. To this end, the organiza-tion adds the membership service provider, which is to providecredentials to clients and peers, to the HLF network. In otherwords, a peer is affiliated with at least one organization toserve as a valid node in the HLF network.
5) Chaincode:
The chaincode (i.e., smart contract) is aprogram, which defines several functions and business logicsto modify retained data stored in the form of key-value pairsas well as to manage the ledger. In general, chaincodes areisolated for data integrity to protect themselves against unau-thorized changes. An endorsing peer can invoke the chaincodeto simulate the business logic that a client specified in atransaction proposal.
6) Block Size:
The block size is one of the configurableparameters of the ordering service. This parameter definesthe maximum number of transactions in one block, whenendorsed transactions sent by clients stack up in the queue inthe ordering service. A new block including the transactionsis promptly exported from the ordering service the momentthat the number of transactions in the queue reaches the pre-defined block size value.
7) Block-generation Timeout:
The block-generation time-out is also one of the configurable parameters of the orderingservice. This parameter defines the maximum time to waitfor other transactions to generate a new block. When a firsttransaction arrives at the empty queue in the ordering service,a timer is identically set to the pre-defined block-generationtimeout value. Once the timer expires, the ordering servicegenerates a new block with all of the waiting transactions,even if the block affords to possess more transactions. Thisparameter prevents transactions from consuming too muchtime in the queue, in case that the number of new transactionsdelivered to the ordering service is insufficient to reach theconfigured block size value.
Smart Home IoT Network Temperature Sensor IoT NetworkDevice Status
IoT Network
Ordering Service ⑤ ⑥⑦
Organization 𝓐 Organization 𝓑 𝓐 . peer.0 𝓐 . peer.2 𝓐 . peer.1 𝓑 . peer.0 𝓑 . peer.1 𝓑 . peer.2 ② ⑨ Hyperledger Fabric for IoT devices ①③ ④ ⑧ HealthcareIoT Network
Base Station Access Point
Base Station
Access Point
Fig. 1: The structure of HLF-based IoT networks
B. Transaction Flow
Transactions in HLF are processed through three phasesto be appended: endorsing phase, ordering phase, and val-idation phase. Each phase has characteristic operations andobjectives, which are essential not only to keep blockchainsystems tamper-proof, but also to ensure data integrity. Inother words, the transaction flow is an important part tosustain a HLF blockchain network. In this subsection, wedescribe the transaction-processing stages and provide detailedexplanations of the three-phased flow.
1) Endorsing Phase:
The endorsing phase is the first phasethat a transaction proposal from the client enters. The mainobjective of this phase is to receive endorsements as proposalsimulation results for the next step. In order to successfullypass this phase, the client needs to ask the sufficient number ofendorsing peers to simulate the transaction proposal, accordingto the endorsement policy. Note that the endorsement policyspecifies by which endorsing peers transaction proposals haveto be endorsed [22]. If a transaction goes to the next phasewithout sufficient endorsements, it is marked as invald in thevalidation phase.When the endorsing peers receive the proposal, they eachsimulate the request against their local status using the spec-ified chaincode. In consequence, they respond to the clientwith endorsements, containing a read/write set. The read setincludes the key to update and its latest version, which alreadyhas been committed to ledger previously, while the write setincludes the key to update and its new value, with whichthe client hopes to update the value of the key. Note thatthis phase does not lead to a ledger update. The client, then,investigates that whether the delivered endorsements have exactly the same simulation results (i.e., the new value of thekey) from one another. If they are identical, the endorsementsare collected in the form of a transaction, and transmittedto the ordering service for the ordering phase. On the otherhands, if they have different simulation results, the clientdiscards the endorsements and refuses to continue processingthe transaction proposal.
2) Ordering Phase:
The ordering phase is the secondphase, in which every delivered transaction is ordered chrono-logically per HLF channel in the ordering service. The mainobjective of this phase is to collect transactions and to producenew blocks. The ordering service amasses all transactions fromevery registered HLF channel separately for data confidential-ity, then generates a new block for each HLF channel. Thenew block is transmitted to the corresponding channel for thevalidation phase. The ordering service is a cluster composedof several nodes, which are eligible for performing the pre-defined consensus protocol. In this paper, we focus on theKafka/ZooKeeper-based ordering service, which is currentlythe most widely used implementation method.
3) Validation Phase:
The validation phase is the last phase,in which the new block is validated by each peer individually.A peer that has received the block conducts two verificationson it sequentially in this phase: (1) Validation system chain-code (VSCC) verification and (2) Multi-version concurrencycontrol (MVCC) verification. The VSCC verification is tomake sure that a transaction has been endorsed by properendorsing peers. If a transaction does not hold appropriateendorsements, it is prohibitted from updating the ledger.Therefore, on the basis of the endorsement policy defining bywhich endorsing peers transactions have to be endorsed for commitment, every transaction in the block is successivelyinvestigated and marked as either valid or invalid in thisverification stage.The MVCC verification is to investigate if the key versionscaptured during the endorsing phase still remain the same asthose in the current ledger locally owned by the peer [20].If they are the same as each other, the transaction is allowedto update the ledger with the data, and the key version ischanged. In contrast, the transaction is considered as invalidif any of the previously conveyed transactions has successfullyupdated the identical data (i.e., the same key-value) ahead ofit, because a key-version of data change happens each timethe data is successfully updated [22].
C. HLF-based IoT Networks
Figure 1 demonstrates HLF-based IoT networks equippedwith a HLF network preserving data generated from IoTdevices. A group of IoT devices can establish a specific IoTnetwork, while HLF facilitates IoT networks to exploit theblockchain storage for data mangement. As can be seen in Fig.1, IoT devices in the smart home and healthcare IoT networksare currently activated to transmit their data to each accesspoint (AP) that they are connected to.In the smart home IoT network, for example, the smartappliances transmit data, which is necessary to be stored to theblockchain system, then AP conveys the delivered data to thenearest base station (BS), then the BS entitled to access theHLF network starts to communicate with the correspondingendorsing peer (i.e., A .peer.0) in order to append the data tothe ledger as a client. Note that IoT devices are consideredto be incapable of joining the HLF network or working asa client by themselves because they may not have efficientresources to proceed with the authentication processes and tomaintain certificates. The proposal sent from the BS is receivedby the endorsing peer, and the endorsing peer simulates thedata against its own ledger, using the smart contract. Uponcompletion of the transaction simulation process, the endorsingpeer returns the result to the BS. Note that if the endorsementpolicy appoints multiple peers to endorsing peers, the BSis required to ensure that all returned results are identicalto one another. After the BS delivers the simulation result(i.e., endorsement) to the ordering service in the form of atransaction, the ordering service amasses endorsed transactionsand creates a new block per HLF channel. Note that theordering service can assort collected transactions, dependingon their sources for privacy by constructing additional HLFchannels. The new block including the transaction is conveyedto the peers belonging to the corresponding channel for thevalidation phase, then the block is finally committed to theledger if it is successfully marked as valid. D. Latency Types in HLF
In Section III.C, we instantiate HLF-based IoT networks,where data from the smart home IoT network is committedto the ledger in order. Before modeling the HLF latency, wefirst divide the total elapsed time of the data in the HLFnetwork according to the transaction-processing stages, then provide the technical definition of each latency type, namely,endorsing latency, ordering latency, validation latency, andledger-commitment (i.e., total latency). Note that we excludethe communication latency from IoT devices to a BS not forspecific cases, but for a wide range of applicable use cases. • The endorsing latency refers to the time taken to askendorsing peers to endorse a transaction and to receivetheir responses. To be specific, it is defined in this paperas the sum of latency 1, 2, 3, and 4 in Fig. 1. • The ordering latency refers to the time taken to receivethe transaction and to await until exported as a new block.It is defined in this paper as the sum of latency 5, 6, 7, and8 in Fig. 1. Note that latency 6 consists of the consensustime and the waiting time for other transactions to bereleased as part of a new block from each transaction’slevel. On the other hand, the latency 8 is the transmissiontime and waiting time of the new block for the validationphase. Therefore, it may be considered as part of thevalidation latency, but is included in the histograms inFigs. 2 and 3. Note that the latency 8 may be highly likelyto work as a performance bottleneck if the previous blockrequires much time to be validated. • The validation latency refers to the time taken to validateand commit the new block. Transactions in the sameblock are sequantially validated, but committed all to-gether as a batch. Therefore, technically speaking, theyall have the identical validation latency from one another.Note that the validation latency is not measured at acommitting peer, but is at an endorsing peer in this paper.To be specific, this latency is defined in this paper aslatency 9 at an endorsing peer in Fig. 1. • The ledger-commitment latency (i.e., total latency) refersto the time taken to completely process a transaction fromthe beginning. It is defined in this paper as the sum ofthe three latency types explained above.IV. HLF L
ATENCY M ODELING
Using probability distribution fitting techniques, we fit acurve to each latency histogram of the endorsing latency,ordering latency, validation latency, and ledger-commitmentlatency. We then find it possible to define their histograms asexisting probability distributions. The probability density func-tions (PDFs) and cumulative distribution functions (CDFs) ofthree probability distributions, which are determined as best-fit distributions through our experiments, are first introduced.Note that the best-fit distribution refers to the probabilitydistribution resulting in the smallest difference from a captureddata set produced from repeated observations out of candi-dates. Subsequently, we not only provide a ledger-commitmentlatency model along with goodness-of-fit test results, but alsoreveal some cases, in which the best-fit distribution for theledger-commitment latency does not work.Even though it is claimed that throughput is more importantthan latency in [16], the significance of latency does not needto be devalued because latency and throughput can not onlycorrelate with each other, but also influence comprehensiveperformance. For example, in age of information (AoI), which (a) (b) (c)
Fig. 2: Ordering latency and ledger-commitment latency histograms from some scenarios, where the probability distributionfitting method based on gamma distributions is not effective.has been recently proposed to quantify and estimate data fresh-ness, both latency and throughput play a key role. Besides,in terms of reliability, even if two different parameter setupsresult in the identical throughput from each other, it is difficultto believe that they are all reliable because either of themmay have large variance among each transaction’s latency[23]. Thus, the significances of latency and throughput areequivalent.
A. Experimental Setup
For the implementation of our HLF network, we exploit theHLF v1.3 sample network provided at [21] on one physicalmachine. The physical machine is with Intel(R) Xeon W-2155@ 3.30 GHz and 16 GB RAM. Consequently, we newly bringup a Kafka/ZooKeeper-based ordering service consisting offour kafka nodes (i.e. ordering nodes) and three frontend nodesthat are connected to the Kafka cluster in the network. Notethat the frontend node is not only to inject transactions fromclients into the Kafka cluster, but also to receive new blocksand to distribute them via the gossip protocol. The number ofendorsing peers and committing peers in the HLF network areone and two, respectively. Note that the number of committingpeers may not be able to affect the histogram shape becausethey are not allowed to involve themselves in the transaction-processing steps, but to validate new blocks and retain copiedledgers only.Each test run includes 1,000 transaction proposals to sub-mit, and all transactions to update each stored key-valueset are conveyed to the endorsing peer in the HLF networkone after another, possessing appropriate arguments for the changeCarOwner function defined in the
Fabcar chaincodeat [21]. Each of the frontend nodes has an equal probabilityof being selected to receive a transaction and to distributea new block. Note that every generated transaction updatesa different key-value from each other only once, therefore,the impact of MVCC violation is not considered in thispaper. All transactions are generated at a certain rate persecond, denoted as λ txn . In this experiments, we not onlyvary this transaction generation rate λ txn , but also measurethe endorsing latency, ordering latency, validation latency, andledger-commitment latency in order to analyze the impact of increasing λ txn . Note that the inter-generation times among allgenerated transactions follow an exponential distribution. B. Best-fit Probability Distributions
The HLF network, as described in Section III-D, includesall the latency types (i.e., endorsing latency, ordering latency,validation latency, and ledger-commitment latency). In orderto provide a latency model by fitting curves to the histograms,we first capture at least over 20,000 transaction samplesfor latency histograms, then seek out a best-fit probabilitydistribution for each latency histogram. The discovered best-fitdistributions are introduced with their PDFs and CDFs as inthe following.
1) Exponential Distribution:
The exponential distributionis the probability distribution of time intervals in a Poissonprocess. This distribution has the PDF as: f ( x ) = λe − λx (1)The CDF of the exponential distribution is given as follows: F ( x ) = 1 − e − λx (2)Note that λ is the rate parameter for λ > . In this paper,the exponential distribution is used for probability distributionfitting to the endorsing latency.
2) Gamma Distribution:
The gamma distribution is a twoparameter frequency distribution [24]. The PDF of the gammadistribution is defined for α > , β > as: f ( x ) = β α Γ( α ) x α − e − βx (3)On the other hand, the CDF is given for α > , β > asbelow: F ( x ) = 1Γ( α ) γ ( α, βx ) (4)The gamma function, denoted as Γ( α ) , is defined for α > as follows: Γ( α ) = (cid:90) ∞ x α − e − x dx (5) Fig. 3: The histograms of endorsing latency, ordering latency, validation latency, and ledger-commitment latency, where theblock size is 10, block-generation timeout is 1 second, and transaction generation rate is 10 transactions/second.The lower incomplete gamma function, denoted as γ ( α, x ) ,is defined for α > , x ≥ as follows [25]: γ ( α, x ) = (cid:90) x t α − e − t dt (6)Note that α is the shape parameter, and β is the inversescale parameter (i.e., rate parameter). In this paper, the gammadistribution is used for probability distribution fitting to theordering latency and ledger-commitment latency.
3) Generalized Extreme Value (GEV) distribution:
TheGEV distribution encompasses the three standard extremevalue distributions: Gumbel ( ξ = 0 ), Fr´echet ( ξ > ), andWeibull ( ξ < ) [26]. Depending on the shape parameter,denoted as ξ , the GEV distribution results in one of the threeprobability distributions, which are also respectively referredto as GEV distribution’s Type I, II, and III in the listed orderabove. The PDF of the GEV distribution is given as thefollowing expression for −∞ < x ≤ µ − σξ when ξ < ,for µ − σξ ≤ x < ∞ when ξ > , and for −∞ ≤ x ≤ ∞ when ξ = 0 [27]: f ( x ) = (cid:8) ξ (cid:0) x − µσ (cid:1)(cid:9) − [( ξ ) − ] σ exp (cid:110)(cid:0) ξ (cid:0) x − µσ (cid:1)(cid:1) − ξ (cid:111) , if ξ (cid:54) = 01 σ exp (cid:26) − x − µσ − exp (cid:20) − x − µσ (cid:21)(cid:27) , if ξ = 0 (7)On the other hand, the CDF of the GEV distribution isdefined for −∞ < x ≤ µ − σξ when ξ < , for µ − σξ ≤ x < ∞ when ξ > , and for −∞ ≤ x ≤ ∞ when ξ = 0 as follows: F ( x ) = exp (cid:110) − (cid:0) ξ (cid:0) x − µσ (cid:1)(cid:1) − ξ (cid:111) , if ξ (cid:54) = 0exp (cid:8) − exp (cid:8) − x − µσ (cid:9)(cid:9) , if ξ = 0 (8)Note that µ is the location parameter, and σ is the scaleparameter. In this paper, the GEV distribution is used forprobability distribution fitting to the validation latency. In our (a) λ txn =8 (b) λ txn =9 (c) λ txn =10 (d) λ txn =11 Fig. 4: Ledger-commitment latency model and gamma distributions with varying transaction generation rates, where the blocksize is 10 and block-generation timeout is 2 seconds.
Ledger-commitment latency [sec]
Empirical txn =8Theoretical txn =8Empirical txn =9Theoretical txn =8Empirical txn =10Theoretical txn =10Empirical txn =11Theoretical txn =11
Fig. 5: Empirical CDFs and theoretical CDFs of Fig. 4.observation, validation latency histograms have GEV distribu-tion’s Type II (i.e., the Fr´echet distribution for ξ > ) only. C. Modeling Feasible Conditions
During the modeling process, it is found out that thereexist some cases whose histograms of the ledger-commitmentlatency do not fit into a gamma distribution. In other words,probability distribution fitting based on gamma distributionsis not always effective at configurable HLF parameter setups.We have discovered that two environments, in which a gammadistribution is an inappropriate description of their ledger-commitment latency shapes. Figures 2(b) and 2(c) demonstratetwo ledger-commitment latency histograms measured in suchdifferent setups from each other, which are not well-matchedwith a gamma distribution.As the first example, Figs. 2(a) and 2(b) illustrate orderinglatency and ledger-commitment latency histograms, wherethe block size is 10, block-generation timeout is 1 second,and λ txn is 3 transactions/second, respectively. If the timerin the ordering service expires before the waiting queuefor new transactions becomes full, the collected transactionsare immediately made into a new block, no matter howmany transactions the new block possesses. This implies that when most of generated blocks are influenced by the block-generation timeout parameter, the ordering latency of everyfirst transaction inside each block is almost identical to thepre-defined block-generation timeout value. At this point,if the average number of transactions in one block is toosmall in comparison with the pre-defined block size value,the transactions whose observed ordering latency values arealmost identical to the pre-defined block-generation timeoutvalue are presented in the histogram with high frequency, andmake a gamma distribution ineffective at fitting curves. In fact,the percentage of blocks influenced by the block-generationtimeout is 99.803 in Figs. 2(a) and 2(b). This leaves a sharpincrease around 1.2 seconds in Fig. 2(a) and two peaks around1.2 and 1.6 seconds in Fig. 2(b).On the other hand, if the block-generation rate of the order-ing service is too high that peers cannot handle flooding newblocks promptly, the blocks have high waiting time consumedfor entering the validation phase. Figure 2(c) demonstrates theledger-commitment latency histogram, where the block size is15, block-generation timeout is 0.75 seconds, and λ txn is 10transactions/second. In this case, most of the generated blocksare influenced by the block size parameter because λ txn ishigh enough that new transactions are put into many generatedblocks to the limit. As can be seen in Fig. 2(c), however, agamma distribution is not a good match with the shape. Thus,this paper are not focused on these two cases. D. Latency Modeling and Validation
In Section IV.A, we explain the experimental setup, fromwhich the latency histograms are obtained, then discuss theprobability distribution fitting feasible condition. We nowillustrate the histograms of endorsing latency, ordering latency,validation latency, and ledger-commitment latency, then fitcurves to them. In Fig. 3, when λ txn is 10, the endorsinglatency accords with the exponential distribution. The orderinglatency and ledger-commitment latency match the gammadistributions. Especially, although the ordering latency leavesroom for another best-fit porbability distribution, the ledger-commitment latency is well-matched with the gamma distri-bution. On the other hand, the validation latency works withthe GEV distribution.Figure 4 demonstrates histograms of the ledger-commitmentlatency with each of their best-fit gamma distributions in the TABLE I: Estimated gamma distribution parameters for the theoretical ledger-commitment latency and KS test results withvarying the block-generation timeout, block size, and transaction generation rate values.
Timeout = 1[sec] λ txn [transactions/sec] Timeout = 1[sec] λ txn [transactions/sec]Block size = 20
10 11 12 13
Block size = 25
17 18 19 20
Estimated ( α txn , β txn ) (8.9573, 5.5858) (9.5112, 6.0407) (9.3159, 6.2078) (10.0333, 6.2937) Estimated ( α txn , β txn ) (9.7902, 6.0267) (9.1368, 6.131) (9.005, 5.9462) (8.2069, 5.395) Average KS statistic
Average KS statistic
Empirical latency [sec]
Empirical latency [sec]
Theoretical latency [sec]
Theoretical latency [sec]
Timeout = 2[sec] λ txn [transactions/sec] Timeout = 2[sec] λ txn [transactions/sec]Block size = 10 Block size = 20
15 16 17 18
Estimated ( α txn , β txn ) (7.181, 4.0151) (6.8603, 4.6625) (5.6829, 4.355) (7.1636, 4.1801) Estimated ( α txn , β txn ) (7.6898, 4.6474) (8.4486, 5.1794) (8.0874, 5.2866) (8.3269, 4.8638) Average KS statistic
Average KS statistic
Empirical latency [sec]
Empirical latency [sec]
Theoretical latency [sec]
Theoretical latency [sec]
Timeout = 3[sec] λ txn [transactions/sec] Timeout = 3[sec] λ txn [transactions/sec]Block size = 20
15 16 17 18
Block size = 30
23 24 25 26
Estimated ( α txn , β txn ) (7.9739, 4.9723) (7.608, 4.9499) (7.0015, 4.9697) (5.7078, 4.1482) Estimated ( α txn , β txn ) (8.0788, 4.6915) (7.0546, 4.527) (6.3698, 4.4257) (7.3836, 4.8698) Average KS statistic
Average KS statistic
Empirical latency [sec]
Empirical latency [sec]
Theoretical latency [sec]
Theoretical latency [sec] λ txn range from 8 to 11, and Fig. 5 shows empirical andtheoretical CDFs. Table I shows estimated gamma distributionparameter pairs and goodness-of-fit test results obtained from avariety of HLF environments, including the estimated gammadistribution parameters of Fig. 4. We consider all captureddata during one test run as outliers when their mean ledger-commitment latency value is remarkably longer than those ofthe other test runs. We exclude such outliers in a differentway from [14]. Note that the median empirical results arecompared to the estimated ones in [14]. In Fig. 4 and thecorresponding results table (i.e., the third sub-table in TableI), the mean ledger-commitment latency decreases as λ txn increases, allowing the ordering service to generate new blocksfaster. However, the trend goes in the opposite direction when λ txn becomes 11. This is because some new blocks graduallystart to wait for the validation phase, due to the fast block-generation rate. Note that we leverage nonlinear regression todiscover the best-fit parameters.In order to validate and improve the reliablity of ourlatency model, we demonstrate Kolmogorov-Smirnov (KS)test results for evaluating goodness-of-fit in Table I. Note thatthe estimated parameters and KS statistics are averaged overten test runs at the 0.01 significance level. The critical valueis then set to 0.0513, in other words, the latency model can beconsidered as reliable when the KS statistics value is smallerthan 0.0513. Thus, the test passes with success for most ofthe cases in Table I, furthermore, the theoretical latency isstill analogous to the empirical latency even in the cases thatthe test does not successfully pass.Furthermore, when the average throughput is defined as theaverage number of completed transactions per second, theaverage throughput of HLF can also be estimated. In this paper, the average ledger-commitment latency is defined asfollows: E [ T total ] = (cid:80) Ni =1 t i N (9)where T total is the total latency (i.e., ledger-commitmentlatency) when the block size and block-generation timeout aregiven, N is the total number of transmitted transactions, and t i is the ledger-commitment latency of the i -th transaction.On the other hand, the average throughput is defined asfollows: E [ T total ] = N (cid:80) Ni =1 t i (10)Therefore, the average throughput can be obtained based onthe definition of the average ledger-commitment latency.V. I NFLUENTIAL P ARAMETER A NALYSIS ON L ATENCY
From the viewpoint of an IoT device, which handles time-sensitive jobs and desires faster transaction commitment, theoptimal design of HLF can be achieved by minimizing theledger-commitment latency. As defined in Section III.A, theblock size and block-generation timeout parameters are theimportant parameters to control producing new blocks in theordering service. This implies that the block-generation rate ofthe ordering service, which is highly influential on the ledger-commitment latency and determined by the other parametersindirectly, also can be under control to minimize the ledger-commitment latency. Focusing on the two parameters, there-fore, we first explore their impacts on the ledger-commitmentlatency in order to provide some insights on the methods Block size [number]
Ledge r- c o mm i t m en t l a t en cy [ s e c ] Empirical timeout 2[sec]Theoretical timeout 2[sec]Empirical timeout 4[sec]Theoretical timeout 4[sec]
Fig. 6: Effect of block size on the ledger-commitment latency,where λ txn is 10 and the block-generation timeout is 2 and 4seconds.for reducing and minimizing the ledger-commitment latency.Moreover, we reveal that 1) the optimal block size and block-generation timeout parameter values for the lowest latencyexist, and 2) an increase in the block-generation rate leadsto two conflicting effects on the ledger-commitment latency. A. Impact of Block Size
As defined in Section III.A, the block size parameter is todefine the maximum number of transactions in a block. Figure6 illustrates the impact of the block size on the empirical andtheoretical ledger-commitment latency. Note that we estimateit in a probability distribution fitting feasible range on thebasis of Section IV.C. Before the block size reaches each ofthe optimal values resulting in the lowest ledger-commitmentlatency (i.e., 10 and 12 when the block-generation timeoutis 2 and 4 seconds, respectively), the ledger-commitmentlatency decreases as the block size increases. This is becausethe block-generation rate of the ordering service becomeshigher as the block size is smaller, preventing transactionsfrom staying for a long time in the ordering service waitingqueue. Thus, new blocks are generated and distributed too fastthat the peers cannot promptly handle them. The new blocksgradually stack up in each peer’s queue for the validation withhigh queueing delays (i.e., each block’s waiting time in thelatency 8 in Fig. 1). However, as the block size increases,the ledger-commitment latency decreases and finally becomeslowest due to reduced block-generation rate. Note that theprobability distribution fitting method does not work in thiscase as explained in Section IV.C.When the block size is larger than the optimal value, on theother hand, the ledger-commitment latency increases, then sat-urates shortly. This is because transactions to be included intoa block cannot help waiting longer for others as the block sizeincreases, so the ordering latency increases. Nonetheless, theledger-commitment latency does not increase infinitely despiteof increase in the block size because the timer for the block-
Block-generation timeout [sec]
Empirical block size 10Theoretical block size 10Empirical block size 15Theoretical block size 15
Fig. 7: Effect of block-generation timeout on the ledger-commitment latency, where λ txn is 10 and the block size is10 and 15.generation timeout starts to expire, and waiting transactions areforced into forming a new block without awaiting others untilthe number of waiting transactions becomes identical to theblock size from some point on. Hence, the ledger-commitmentlatency ends up saturating as the block-generation timeout isfixed in Fig. 6. Note that the case when the block-generationtimeout is set to 4 seconds in Fig. 6 requires a much largerblock size for saturation. B. Impact of Block-generation Timeout
The block-generation timeout parameter is to prevent wait-ing transactions to be included into a block from spendinglong time in the ordering service for better performance.Figure 7 demonstrates the impact of the block-generationtimeout on the empirical and theoretical ledger-commitmentlatency. The ledger-commitment latency decreases as theblock-generation timeout increases until the block-generationtimeout reaches each of the optimal values resulting in thelowest ledger-commitment latency (i.e., 2 and 1.25 secondswhen the block size is 10 and 15, respectively). Besides,the ledger-commitment latency reversely increases with theblock-generation timeout for the same reasons as explained inSection V.A.However, different from Fig. 6, the ledger-commitmentlatency slightly decreases and ends up saturating as Fig. 6.We presume that this slight decrease is due to the orderinglatency and validation latency. To be specific, when the block-generation timeout keeps increasing, the queueing delay toenter the validation phase decreases and finally touches zero.On the other hand, the rest of the ordering latency excludingthe latency 8 in Fig. 1 and validation latency increase, althoughthey saturate later. In this sense, when the ledger-commitmentlatency increases with the block-generation timeout, the or-dering latency and validation latency still increase, while thequeueing delay for the validation phase decreases steadily (i.e.,from 2 to 2.75 seconds and from 1.25 to 1.75 seconds when the block size is 10 and 15, respectively). However, the orderinglatency and validation latency start to saturate because of theblock size parameter ahead of the queueing delay, while thequeueing delay for the validation phase decreases a bit longer,then disappears. Thus, it is concluded that each increase inthe block size and block-generation timeout cannot affect theledger-commitment latency infinitely, but makes it saturatewhen the other parameters are fixed.VI. C ONCLUSIONS
In this paper, we propose a latency model of HLF v1.3,which is instantiated as HLF-based IoT networks, where eachspecific IoT network can deposit permanent data in the HLFblockchain network for data integrity and auditability. To thisend, we first implement a HLF network on our physical ma-chine and embed a Kafka/ZooKeeper ordering service into it.In an empirical approach, we construct histograms using cap-tured transaction latency samples, then develop a hypothesisthat the ledger-commitment latency is modeled with a gammadistribution in some configurable setup range. Introducing theKS test to evaluate goodness-of-fit, we prove that our latencymodel is reliable. In the sequel, considering the characteristicthat IoT devices handle latency critical tasks in general, weanalyze the impacts of two influential HLF parameters onthe ledger-commitment latency, in order to provide practicalinsights on reducing and optimizing the ledger-commitmentlatency. We hope that this paper would serve as guidelinesto theoretically estimating the ledger-commitment latency andto implementing HLF with the optimal design for HLF-basedIoT networks. R
EFERENCES[1] A. Dorri, S. S. Kanhere, R. Jurdak, and P. Gauravaram, “Blockchain forIoT security and privacy: The case study of a smart home,” in
Proc. IEEEInt. Conf. Pervasive Comput. Commun. Workshops (PerCom WKSHPS) ,Kona, HI, USA, Mar. 2017, pp. 1–6.[2] T. T. A. Dihn, J. Wang, G. Chen, R. Liu, B. C. Ooi, and K.-L. Tan,“BLOCKBENCH: A framework for analyzing private blockchains,” in
Proc. ACM Int. Conf. Manag. of data , Chicago, IL, USA, May 2017,pp. 1–16.[3] T. T. A. Dinh, R. Liu, M. Zhang, G. Chen, B. C. Ooi, and J. Wang,“Untangling blockchain: A data processing view of blockchain systems,”
IEEE Trans. Knowl. Data Eng. , vol. 30, no. 7, pp. 1366–1385, Jul. 2018.[4] S. Pongnumkul, C. Siripanpornchana, and S. Thajchayapong, “Perfor-mance analysis of private blockchain platforms in varying workloads,”in
Proc. Int. Conf. Comput. Commun. Netw. (ICCCN) , Vancouver, BC,Canada, Aug. 2017, pp. 1–6.[5] P. Thakkar, S. Nathan, and B. Viswanathan, “Performance benchmarkingand optimizing Hyperledger Fabric blockchain platform,” in
Proc. IEEEInt. Symp. Model., Anal., Simul. Comput. Telecommun. Syst. (MAS-COTS) , Milwaukee, WI, USA, Sep. 2018, pp. 1–13.[6] S. Shalaby, A. A. Abdellatif, A. Al-Ali, A. Mohamed, A. Erbad, andM. Guizani, “Performance evaluation of Hyperledger Fabric,” in
Proc.IEEE Int. Conf. Inform., IoT, Enabling Technol. (ICIoT) , Doha, Qatar,Feb. 2020, pp. 1–6.[7] M. Kuzlu, M. Pipattanasomporn, L. Gurses, and S. Rahman, “Per-formance analysis of a Hyperledger Fabric blockchain framework:Throughput, latency, and scalability,” in
Proc. IEEE Int. Conf.Blockchain (Blockchain) , Atlanta, GA, USA, Jul. 2020, pp. 1–5.[8] Q. Nasir, I. A. Qasse, M. A. Talib, and A. B. Nassif, “Performanceanalysis of Hyperledger Fabric platforms,”
Secur. Commun. Netw. , vol.2018, no. 1, pp. 1–14, Sep. 2018.[9] H. Lee, C. Yoon, S. Bae, S. Lee, K. Lee, S. Kang, K. Sung, and S. Min,“Multi-batch scheduling for improving performance of HyperledgerFabric based IoT applications,” in
Proc. IEEE Global Commun. Conf.(GLOBECOM) , Waikoloa, HI, USA, Dec. 2019, pp. 1–6. [10] T. S. L. Nguyen, G. Jourjon, M. Potop-Butucaru, and K. L. Thai, “Impactof network delays on Hyperledger Fabric,” in
Proc. IEEE Int. Conf.Comput. Commun. Workshops (INFOCOM WKSHPS) , Paris, France,Apr. 2019, pp. 1–6.[11] L. Foschini, A. Gavagna, G. Martuscelli, and R. Montanari, “Hyper-ledger Fabric blockchain: Chaincode performance analysis,” in
Proc.IEEE Int. Conf. Commun. (ICC) , Dublin, Ireland, Jun. 2020, pp. 1–6.[12] S. Wang, “Performance evaluation of Hyperledger Fabric with maliciousbehavior,” in
Proc. Int. Conf. Blockchain (ICBC) , San Diego, CA, USA,Jun. 2019, pp. 1–9.[13] T. Nakaike, Q. Zhang, Y. Ueda, T. Inagaki, and M. Ohara, “HyperledgerFabric performance characterization and optimization using GoLevelDBbenchmark,” in
Proc. IEEE Int. Conf. Blockchain Cryptocurrency(ICBC) , Toronto, ON, Canada, May 2020, pp. 1–9.[14] H. Sukhwani, J. M. Martinez, X. Chang, K. S. Trivedi, and A. Rindos,“Performance modeling of PBFT consensus process for permissionedblockchain network (Hyperledger Fabric),” in
Proc. Symp. Rel. Distrib.Syst. (SRDS) , Hong Kong, China, Sep. 2017, pp. 1–3.[15] H. Sukhwani, N. Wang, K. S. Trivedi, and A. Rindos, “Performancemodeling of Hyperledger Fabric (permissioned blockchain network),”in
Proc. IEEE Int. Symp. Netw. Comput. Appl. (NCA) , Cambridge, MA,USA, Nov. 2018, pp. 1–10.[16] P. Yuan, K. Zheng, X. Xiong, K. Zhang, and L. Lei, “Performancemodeling and analysis of a Hyperledger-based system using GSPN,”
Comput. Commun. , vol. 153, no. 1, pp. 117–124, Mar. 2020.[17] L. Jiang, X. Chang, Y. Liu, J. Misic, and V. B. Misic, “Performance anal-ysis of Hyperledger Fabric platform: A hierarchical model approach,”
Peer Peer Netw. Appl. , vol. 13, no. 3, pp. 1014–1025, Jan. 2020.[18] J. Sousa, A. Bessani, and M. Vukolic, “A Byzantine fault-tolerantordering service for the Hyperledger Fabric blockchain platform,” in
Proc. IEEE/IFIP Int. Conf. Dependable Syst. Netw. (DSN)
Proc. Eur. Conf. Comput. Syst. (EuroSys) , Porto, Portugal, Jan. 2018,pp. 1–15.[21] https://hyperledger-fabric.readthedocs.io/en/release-1.3.[22] S. Lee, M. Kim, R.-H. Hsu, and T. Q. S. Quek, “Is blockchain suitablefor data freshness? – Age-of-Information perspective,” arXiv preprintarXiv:2006.02735, to appear in IEEE Netw. , pp. 1–7, Jun. 2020.[23] A. Bruton, J. H. Conway, and S. T. Holgate, “Reliability: What is it,and how is it measured?”
Physiotherapy , vol. 86, no. 2, pp. 94–99, Feb.2000.[24] H. S. C. Thom, “A note on the gamma distribution,”
Mon. Weather Rev. ,vol. 86, no. 4, pp. 117–122, Apr. 1958.[25] G. J. O. Jameson, “The incomplete gamma functions,”
Math. Gaz. , vol.100, no. 548, pp. 298–306, Jul. 2016.[26] T. G. Bali, “The generalized extreme value distribution,”
Econ. Lett. ,vol. 79, no. 3, pp. 423–427, Jan. 2003.[27] W. Huang, S. Xu, and S. Nnaji, “Evaluation of GEV model for frequencyanalysis of annual maximum water levels in the coast of United States,”