Kriptosare.gen, a dockerized Bitcoin testbed: analysis of server performance
Francesco Zola, Cristina Pérez-Solá, Jon Egaña Zubia, Maria Eguimendia, Jordi Herrera-Joancomartí
22019 10th IFIP International Conference on NewTechnologies, Mobility and Security (NTMS) (cid:13) c (cid:13) a r X i v : . [ c s . PF ] O c t riptosare.gen, a dockerized Bitcoin testbed: analysis of serverperformance Francesco Zola ∗ , Cristina P´erez-Sol`a † , Jon Ega˜na Zubia ∗ , Maria Eguimendia ∗ , Jordi Herrera-Joancomart´ı ‡∗ Dept. Data Intelligence for Energy and Industrial Processes, Vicomtech,Paseo Mikeletegi 57, 20009 Donostia/San Sebastian, Spain { fzola, jegana, meguimendia } @vicomtech.org † Internet Interdisciplinary Institute (IN3), Universitat Oberta de Catalunya (UOC),CYBERCAT-Center for Cybersecurity Research of Catalonia, 08860 Castelldefels, Barcelona, [email protected] ‡ Dept. d’Enginyeria de la Informaci´o i les Comunicacions, Universitat Aut`onoma de Barcelona,CYBERCAT-Center for Cybersecurity Research of Catalonia, 08193 Bellaterra, Catalonia, [email protected]
Abstract — Bitcoin is a peer-to-peer distributed cryptocur-rency system, that keeps all transaction history in a publicledger known as blockchain. The Bitcoin network is implicitlypseudoanonymous and its nodes are controlled by independententities making network analysis difficult. This calls for thedevelopment of a fully controlled testing environment.This paper presents
Kriptosare.gen , a dockerized automa-tized Bitcoin testbed, for deploying full-scale custom Bitcoinnetworks. The testbed is deployed in a single machine executingfour different experiments, each one with different networkconfiguration. We perform a cost analysis to investigate howthe resources are related with network parameters and provideexperimental data quantifying the amount of computationalresources needed to run the different types of simulations.Obtained results demonstrate that it is possible to run thetestbed with a configuration similar to a real Bitcoin system.
Index Terms — Bitcoin testbed, blockchain , dockerized net-work, server performance, regtest
I. INTRODUCTION
Bitcoin uses a P2P network to propagateall the information within the system[10].Peers broadcast their transactions to otherpeers through this network and miners useit to propagate their newly found blocks.Bitcoin is implemented using a decen-tralized peer-to-peer network and, becauseof that, there is no central point of failurewhere the network can be attacked. How-ever, it also hinders the analysis of the net-work. Bitcoin nodes only have a partial localview of the network, so there is no directway to analyze the network as a whole. Since nodes are controlled by independententities, there is no way to know their exactbehavior. Although most nodes run standardversions of the Bitcoin Core client , manyalso run modified versions, either for re-search or for ideological purposes. In anycase, the exact responses and behaviour ofa node with respect to certain network mes-sages and in specific situations have beenused to attack the network in the past, forinstance, by fingerprinting users [2] or byrevealing the network topology [8], [4]. Notbeing able to see the whole network as wellas not knowing how nodes behave, makesanalysis of the Bitcoin network very diffi-cult.The contribution of this paper is to present Kriptosare.gen , a platform for deployingfull-scale Bitcoin networks, together withthe tools to control the network.
Krip-tosare.gen provides automatized featuresthat allow users to deploy a large numberof Bitcoin nodes and simulate the Bitcoinnode interactions in an easy way. This au-tomatic process increases the scalability of https://bitnodes.earn.com/nodes/ he testbed.Is also important to understand underwhich conditions is possible to execute asimulation within Kriptosare.gen . For thisreason, the paper provides data quantify-ing the amount of computational resourcesneeded to run different simulations. Theseresults allow researchers to estimate the costof running a simulation, and provide boundson the simulations that can be executed in agiven hardware.With a tool that is able to reproduce theBitcoin network in a fully controlled en-vironment where all peers are monitored,and where data from all of them can beincorporated into the analysis, researchersand developers are able, for instance, tostudy how changes of the protocol affect thenetwork or to simulate network attacks oruser behaviour.The rest of the paper is organized as fol-lows. Section II describes the related work.After that, Section III presents the archi-tecture of the designed testbed. Then, Sec-tion IV provides experimental evaluation ofthe testbed. Finally, Section V concludes thepaper and provides guidelines for furtherwork.
II. STATE OF THE ART
Simulation allows to create environmentsto experiment with Bitcoin networks with-out actually deploying them.Simulators provide a good approach torecreate a real environment since they areoften developed and optimized in order tosatisfy a simulation goal. As a result, re-quired hardware resources can be bounded,and large simulations can be performed.However, since simulators are specializeddeveloped software (normally derived from the software that runs the real system) theyneed to evolve in parallel to the real soft-ware they simulate. Such a task becomesvery time demanding in highly dynamicsoftware, like Bitcoin, as it is pointed outin [11]. To overcome that problem, anotherapproach is to set an emulation environmentusing the standard software that the originalsystem is using. To that end, bitcoind (thereference Bitcoin client, that implements thefull Bitcoin protocol) has a regression testmode or regtest that allows to createprivate local Bitcoin networks for testingand development purposes. A bitcoind clientrunning in regtest mode will follow thesame rules than a testnet node, but willwork on a new private blockchain (with anew genesis block). The main property thismode offers is the ability to arbitrarily createblocks, that is, blocks are created on de-mand, by issuing a special remote procedurecall (RPC) command to the node, allowingdevelopers to control the network. However,the regtest mode just provides a way fora Bitcoin client to work on a private networkwith a private blockchain, but does not offerany tools to deploy such a network, generatetraffic and replicate common behaviors ormonitorize and visualize its status.Arthur Gervais et al. created a Bitcoin sim-ulator [5] for ns3 [13] (a popular networksimulation) focused on reproducing blockpropagation through the network. Since theywere studying block propagation, the simu-lator is just limited to blocks, and does notreproduce any other Bitcoin network events(propagation of transactions, network mes-sages, etc.). A. Miller and R. Jansen imple-mented a Bitcoin simulator [7] for Shadow,that allows to run real applications over aimulated Internet topology.There exist some basic private Bitcoinnetwork generation platforms based onDocker [9], [6], [1]. Nevertheless, many ofthem are very similar to each other in termsof implementation, lack a clear explanation,and are based on just few nodes (usually2). Although these projects present a pro-cedure to automatize the network creation,they do not consider the network topologynor scalability problems. Other basic privateBitcoin network generations [12], [14] donot provide automatization for user simu-lation, which means that the operations ofthe single Bitcoin user have to be executedmanually. Therefore, they require a lot ofoperations/effort to create a large numberof nodes and to replicate user interactionswithin the Bitcoin network. To the best ofour knowledge, these projects do not offerenough information (like needed resources)to execute a large number of nodes nor toolsto monitor the network’s state.This paper fills in this gap by present-ing a complete automatized Bitcoin networktestbed platform that allows deployment, in-teraction, and resource monitoring of privateBitcoin networks.
Kriptosare.gen allows re-searchers to deploy a full-scale Bitcoin net-work, control the topology of that network,and automatically simulate user behaviour.
III. ARCHITECTURE
A. Deploying the network
The
Kriptosare.gen testbed consists of aprivate Bitcoin network that replicates thereal architecture on-scale. Its deploymentis based on Docker, a platform that allowscreating ”containers” with all the requiredsoftware. Containerized software will al-ways run the same code, regardless of the environment, ensuring high portability andeasy setup. The testbed creates containersas many as there are bitcoin users, so thateach container represents a real user/wallet.In its basic setting, each node runs a BitcoinCore client instance in regtest mode rep-resenting a wallet in the real world.Bitcoin clients inside each of the dockercontainers are controlled by the testbedthrough remote procedure calls sent to their port. Additionally, each bitcoindclient in each container uses port to establish connections with other nodes inthe P2P network. The regtest mode doesnot include a peer discovery mechanism, sonodes do not automatically try to connectto other nodes. Therefore, connections be-tween nodes in the testbed are created byexplicitly sending RPC commands to nodes,and thus
Kriptosare.gen is able to controlthe topology of the network.RPC calls to each bitcoind instance arealso used to generate activity in the testbed.On the one hand, in order to generatetransactions, the testbed creates a thread,that periodically sends transaction genera-tion RPC commands to nodes in the testbed.Specifically, source and destination nodesare chosen randomly, and the amount totransfer is also selected randomly from thebalance available in the source node’s wal-let. A new address is then requested to thedestination node, and a transaction is sent tothis newly generated address.On the other hand, blocks are generatedby randomly selecting one of the bitcoindinstances as the next miner, and sending itthe proper RPC call. . Resources monitoring and storage
In order to evaluate the resources beingused by
Kriptosare.gen in realtime, we con-sider five different metrics. These metricsare collected by Telegraf, an agent for col-lecting and reporting metrics and events.The analyzed metrics collected from Tele-graf to evaluate the performance of ourtestbed are: • CPU usage percentage : the percentageof CPU used by the testbed. • Memory usage : the amount of RAMmemory used by the testbed (GB). • Disk usage percentage : the percentageof disk space needed to run the simula-tion. • Disk I/O : the speed of the host serverduring the operation on a physical disk(read/write) (MB/s). • Network speed : speed of network trafficwithin the whole testbed (KB/s).The first three metrics are computed di-rectly by Telegraf. The last two are calcu-lated from the amount of data write/read andsent/received (respectively, for Disk I/O andfor Network speed), deriving the samplesprovided by Telegraf over time.Data from the mentioned metrics arestored to an InfluxDB , and then processedand visualized in Python, inside a JupyterNotebook. InfluxDB is an open-sourcedatabase suited for time series (TSDB).Once the database is created, the data arestored in data points, each of them madeby a measurement, a tagset, a fieldset, anda timestamp. Data from InfluxDB are thenread from a python Jupyter Notebook, andprocessed to extract all the information fromthe time range of the simulations. Batches of simulations are processed to obtain theaverages and standard deviations of each ofthe selected metrics. IV. P
ERFORMANCE EVALUATION
The amount of resources needed to run asimulation with
Kriptosare.gen depends onthe parameters of the simulation one wantsto run. In this section, we study the impacteach of the parameters of the simulationhas on the required server resources whendeploying the testbed on a single machine.In order to evaluate the performance ofthe testbed when it is executed on a singlemachine, we run a set of simulations . Eachsimulation is defined by its configuration ,that is, a set of parameters together with thevalues they take in the simulation. The word batch is used to indicate a set of simulationswith the same configuration. Simulationsare repeated five times with the same con-figuration to ensure that reported metrics arerepresentative enough. Then, the resourcesneeded to run the simulations are evaluatedwith respect to a set of metrics (as detailed inSection III-B). For each metric, we provideaverage and standard deviations of all thesimulations in a batch.The parameters used in the configurationsare: the number of Bitcoin nodes, the num-ber of peers each node has within the Bit-coin network, the speed at which trans-actions are generated in the network (TXspeed), and the speed at which blocks aremined in the network (BLK speed).We define an experiment as a set of sim-ulations that tries to evaluate the impact ofa given parameter in the resources neededto run the testbed. An experiment has threeof the four parameters of the configurationfixed to a constant value, and the fourth xp. Id
TABLE I: Summary of the configuration of each experiment varied within an interval. This allows us toevaluate the impact of the varied parameteron the resources needed to run the simula-tion.All the simulations are made in a virtualmachine with 4 CPUs Intel(R) Xeon(R) Sil-ver 4114 CPU @ 2.20GHz, 32GB RAMDDR4 memory with 2,666 MHz, and100GB of Hard Disk SATA.Table I presents a summary of the configu-rations of the experiments, where TX speedis measured in transactions per second andBLK speed in blocks per hour. Experimentshave been created in an increasing order ofcomplexity, starting with a basic configura-tion where no peers, transactions or blocksare created, and finishing with a realisticconfiguration where nodes have peers, gen-erate transactions, and mine blocks.
A. Experiment 1: Number of nodes
In the first experiment, we evaluate theimpact of increasing the number of Bitcoinnodes in the network. Therefore, the numberof Bitcoin nodes in the simulation is varied,starting from 0 up to 400. Nodes in thisexperiment do not have any connections toother peers of the network, nor generate anytransactions nor blocks. Therefore, this ex-periment allows to account for the resourcesneeded to create docker containers with bit-coind nodes.Each point in the figures of this sectionshows the average and standard deviationof the metric as reported in a batch of five (a) CPU usage (b) Memory usage(c) Disk usage (d) Disk I/O
Fig. 1: First experiment results: increasing the number ofnodes simulations with the same configuration.This experiment uses around of CPU,with an slow increase with the number ofnodes (see Figure 1a). The consumption ofRAM memory increases linearly with re-spect to the number of bitcoin nodes in thesimulation (Figure 1b). of disk spaceis used by the testbed platform (withoutany deployed nodes), and disk usage in-creases with each new container deployed(Figure 1c).Up to 400 nodes can be created with thehardware resources allocated for the exper-iment; the amount of RAM memory usedbeing the limiting factor.
B. Experiment 2: Number of connections per node
In the second experiment, we evaluatethe resources needed to run the simulationswhen there are Bitcoin nodes and whenthose nodes are connected to each other.Therefore, the testbed now needs to main-tain the connections each node has, andmanage and process the network traffic thatis sent through these connections.All the simulations in this experiment havethe number of nodes fixed to 100. The num-ber of peers per node, ranges between 2nd 16 (tested values are { , , , , } ).The network topology for simulations in thisexperiment is fixed, with each peer in thenetwork having exactly the same number ofconnections. (a) CPU usage (b) Memory usage(c) Disk usage (d) Disk I/O(e) Network speed Fig. 2: Second experiment results: increasing the number ofpeers per node
Changing the number of connection be-tween the nodes in the network has littleimpact on the CPU (Figure 2a), memory(Figure 2b), and disk usage (Figure 2c).Note that the metrics shown in the plots ofthis experiment are evaluated after the con-tainers have been created. Therefore, theyaccount only for the resources needed tocreate and maintain the network connections(and maintain the docker containers), butdo not take into account the cost of creat-ing these containers. This is why CPU andmemory usage reported in this experimentis lower than in Experiment 1. We decidednot to consider container generation here inorder to evaluate the impact of increasingthe number of connections in the resourcesneeded to run the simulation. Increasing the number of connections pernode increases the speed of network traf-fic in the simulation (Figure 2e). However,this increase is really small. The reasonis that these connections are barely used,since there are no transactions nor blocks topropagate. This is also why increasing thenumber of peers per node has also minimalimpact on the memory needed to run thesimulation (Figure 2b): since no transac-tions nor blocks are propagated, there is noneed to perform costly cryptographic opera-tions such as hashes, signature generations,and signature validations.
C. Experiment 3: Transaction generation speed
In the third experiment, we evaluate theperformance of the testbed when transac-tions are generated and propagated throughthe network at different speeds.In this batch, the simulations have thenumber of the nodes fixed to 100 and thenumber of connections fixed to 8. Duringthis experiment, no blocks are yet created,so all transactions remain in the mempool ofthe nodes. The transaction generation speedranges between and tx/s (tested valuesare { , , , , } ). Like in the previousexperiment we do not consider the resourcesneeded to create the containers.Disk and memory usage are not af-fected by transaction generation speed (seeFigs. 3b and 3c). On the contrary, CPUusage shows a nearly lineal increase withthe increment on tx generation speed. Thechange of tendency shown in the highestspeeds is due to the procedure used to gen-erate transactions. In our testbed, increasingtx speed implies increasing the number ofthreads that manage tx generation and, inturn, increases the execution time of a single a) CPU usage (b) Memory usage(c) Disk usage (d) Disk I/O(e) Network speed (f) Tx speed (tx/s) Fig. 3: Third experiment results: increasing transaction speed
RPC call. This behaviour slows down thetx speed, creating a small difference be-tween the nominal value (the speed chosena priori) and the real value (as shown inFigure 3f).Increasing the tx speed increases CPUusage because generating transactions andpropagating them bears computational ef-fort in terms of cryptographic operations.Transactions in the simulation spend a sin-gle P2PKH output and create a singleP2PKH output, and thus one signature hasto be created for each generated transac-tion, and one signature validated for eachreceived transaction.Additionally, the hash of the transaction isalso computed in order to be able to gen-erate and validate the signature. Increasingtransaction generation speed also increasesthe network speed (Figure 3e).
D. Experiment 4: Block generation speed
In the last experiment, we evaluate the per-formance of the server with respect to block generation speed. The number of nodes is , the number of connections per peeris , the transaction generation speed is tx/s, and block generation speed is variable,ranging from to blocks/h (tested val-ues are { , , , , } ). Like the previoustwo experiments we do not consider theresources needed to create the containers.Block generation speed does not affectneither memory nor disk usage (Figs. 4band 4c). Both metrics are stable, and showthe same values than in the previous experi-ment.Network speed shows a slight increase asblock generation speed is incremented (Fig-ure 4e). Comparing this plot with Figure 3e(from Exp. 3 with tx/s), we can observethat the speed is the same, so the blockgeneration speed does not significantly af-fect the network speed. The reason of theseresults is that from Bitcoin Core v0.13.0 on-ward, bitcoind implements compact blockrelaying [3], a protocol for block relayingthat does not re-send transactions in a blockwhenever the peer does already have thesetransactions.Disk I/O speed shows a slight increasewith block speed (Figure 4d), since moreblock metadata has to be written. V. C
ONCLUSION AND FUTURE WORK
The nature of Bitcoin hinders its evalua-tion since, for security reasons, some param-eters are only known at a local level. Hence,broader analysis cannot be performed with-out controlling all elements of the system.For instance, the network discovery mech-anisms used for nodes to join the P2P net-work should provide a highly connected net-work with a difficult to estimate topology.The correctness of such network discovery a) CPU usage (b) Memory usage(c) Disk usage (d) Disk I/O(e) Network speed
Fig. 4: Fourth experiment results: increasing block generationspeed mechanism cannot be properly addressedwithout knowing the exact topology of theproduced network. The work proposed inthis paper goes towards providing new toolsfor a global analysis of Bitcoin that wouldallow to better measure its performance andsecurity.This paper presents
Kriptosare.gen , atestbed based on Docker that allows de-ploying a Bitcoin network in order to em-ulate the real network behavior within acontrolled environment. This testbed givesresearchers full control over the generatednetwork. We have also presented and analy-sis of the cost of running simulation, takinginto account different computer resources.Our results show that RAM and Disk us-age are a limiting factor during the networkcreation phase. The experiments also showthat network traffic increases with transac-tion generation speed in a linear fashion,and that transaction speed also affects CPUusage. Additionally, our results show that in a configuration similar to the real Bitcoinsystem (6 blocks/h) and with a throughputof 7 tx/s, an instantiation of 100 nodes canbe run in a server with 4 CPU, 32GB RAMand 100GB of disk memory.As future work, we intend to deploy thetestbed over multiple servers in order tointroduce routing, latency, and packet lossesproblems, and we will also simulate userbehaviour and improve algorithms to detectundesired actions. The testbed presented inthis paper is a starting point for a multi-currency simulator, so the final idea is toadd new private blockchains based on othercryptocurrencies like Zcash and Monero.
ACKNOWLEDGMENT
This work was partially funded by theEuropean Commission through H2020 pro-gram, of which it is part “TITANIUM”project (grant agreement No 740558). C.P´erez-Sol`a is currently affiliated to UOC,but part of this work was done while work-ing at Universitat Aut `onoma de Barcelonaand Universitat Rovira i Virgili. R EFERENCES[1] Ananichev, D.: Starting the testnet-box. https://github.com/cryptochain/bitcoin-regtest (2017)[2] Biryukov, A., Pustogarov, I.: Bitcoin over Tor isn’t a good idea. In:IEEE Symp. on Security and Privacy (SP). pp. 122–134. IEEE (2015)[3] Corallo, M.: BIP 152. https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki (2016), online; accessed29 January 2019[4] Delgado-Segura, S., Bakshi, S., P´erez-Sol`a, C., Litton, J., Pachulski,A., Miller, A., Bhattacharjee, B.: Txprobe: Discovering bitcoin’snetwork topology using orphan transactions. Proceedings of FinancialCryptography and Data Security (2019)[5] Gervais, A., Karame, G., et al.: On the security and performance ofproof of work blockchains. In: Proc. of the 23nd ACM SIGSAC Conf.on Computer and Communication Security. ACM (2016)[6] Lavine, S.: bitcoin-testnet-box. https://github.com/freewil/bitcoin-testnet-box (2018)[7] Miller, A., Jansen, R.: Shadow-bitcoin: Scalable simulation via directexecution of multi-threaded apps. IACR ePrint , 469 (2015)[8] Miller, A., Litton, J., et al.: Discovering bitcoins public topology andinfluential nodes (2015)[9] Miyamoto, J.: bitcoin-dev-box. https://github.com/joemphilips/bitcoin-dev-box (2017)[10] Nakamoto, S.: Bitcoin: A peer-to-peer electronic cash system (2008)[11] Neudecker, T.: Security and Anonymity Aspects of the Network Layerof Permissionless Blockchains. Ph.D. thesis, KIT (2019)12] Paans, F.: Multinode / Multiwallet Bitcoin regtestnetwork. https://github.com/FreekPaans/bitcoin-multi-node-regtest (2018)[13] Riley, G.F., Henderson, T.R.: The ns-3 network simulator. In: Model-ing and tools for network simulation, pp. 15–34. Springer (2010)[14] de Vroom, A.: Black Flag. https://github.com/ariejan/black-flaghttps://github.com/ariejan/black-flag