Smart Contract Security: a Practitioners' Perspective
Zhiyuan Wan, Xin Xia, David Lo, Jiachi Chen, Xiapu Luo, Xiaohu Yang
SSmart Contract Security: a Practitioners’ Perspective
Zhiyuan Wan ˚: , Xin Xia ;} , David Lo § , Jiachi Chen ; , Xiapu Luo ¶ , Xiaohu Yang ˚˚ College of Computer Science and Technology, Zhejiang University ; Faculty of Information Technology, Monash University § School of Information Systems, Singapore Management University ¶ Department of Computing, Hong Kong Polytechnic University { wanzhiyuan,yangxh } @zju.edu.cn, { xin.xia,jiachi.chen } @monash.edu, [email protected], [email protected] Abstract —Smart contracts have been plagued by securityincidents, which resulted in substantial financial losses. Givennumerous research efforts in addressing the security issues ofsmart contracts, we wondered how software practitioners buildsecurity into smart contracts in practice. We performed a mixtureof qualitative and quantitative studies with 13 interviewees and156 survey respondents from 35 countries across six continentsto understand practitioners’ perceptions and practices on smartcontract security. Our study uncovers practitioners’ motivationsand deterrents of smart contract security, as well as how securityefforts and strategies fit into the development lifecycle. We alsofind that blockchain platforms have a statistically significantimpact on practitioners’ security perceptions and practices ofsmart contract development. Based on our findings, we highlightfuture research directions and provide recommendations forpractitioners.
Index Terms —Security, Empirical study, Smart contract, Prac-titioner
I. I
NTRODUCTION
Blockchain is a distributed ledger that provides anopen, decentralized, and fault-tolerant transaction mechanism.Blockchain technology has attracted considerable attentionfrom both industry and academia since it is originally intro-duced for Bitcoin [40] to support the exchange of cryptocur-rency. Blockchain technology evolves to facilitate general-purpose computations with a wide range of decentralizedapplications. The
Smart contract technology is one appealingdecentralized application that enables the computations on topof a blockchain.A smart contract is a piece of executable code that runs ona blockchain to enforce the terms of an agreement betweenuntrusted parties. Blockchain technology assures that a smartcontract is immutable and contract initiated transactions areautonomously and truthfully executed. There are multipleblockchain platforms that support smart contracts [67], e.g.,Ethereum, Hyperledger Fabric, and Corda, with Ethereumbeing the most prominent platform [4].During the last decade, smart contracts have been plaguedby security incidents, which led to losses reaching millions ofdollars [64]. In June 2016, an attacker exploited vulnerabilitiesin the DAO smart contract to empty out around 4 millionEthers (worth around 50 million dollars). In July 2017, over150 thousand Ethers (worth over 34 million dollars) had been : Also with PengCheng Laboratory. } Corresponding author. stolen due to an exploit in widely-used Parity Wallet [47]. InNovember 2017, over 500 thousand Ethers (worth over 150million dollars) were frozen due to a vulnerability in the verysame wallet [48].To address the security issues of smart contracts, researchershave proposed a broad range of defense solutions, includinglanguage-based security (e.g., [20], [16], [11]), static anal-ysis (e.g., [38], [64], [63]), and runtime verification (e.g.,[56]). Vyper [20] removes some of the language function-alities in Solidity to eliminate vulnerabilities and adds newfeatures to support security and readability. In terms of staticanalysis, Oyente [38] leverages symbolic execution to tra-verse feasible execution paths on control flow graphs anddetect vulnerabilities in smart contracts; Securify [64] definescompliance and violation patterns based on known vulnera-bilities; SmartCheck [63] translates smart contract code intoan XML-based parse-tree and check the parse-tree againstspecific XPath patterns. With respect to runtime verification,Sereum [56] uses taint analysis to monitor runtime data flowsduring the execution of smart contracts for preventing re-entrancy attacks.Despite numerous efforts in assuring the security of smartcontracts, little is known about how software practitionersbuild security into smart contracts in practice. Thus, wefollowed a mixed methods approach to investigate the practi-tioners’ perceptions and practices with respect to smart con-tract security. We started with semi-structured interviews with13 software practitioners with experience in smart contractdevelopment, who have an average of 6.5 years of softwareprofessional experience. Through the interviews, we qualita-tively investigated the security awareness and practices that ourinterviewees experienced in smart contract development. Wederived 6 competing priorities in smart contract development,a list of 11 security motivators and 9 security deterrentsfor smart contract practitioners, 5 sources where practitionersacquire security knowledge, 11 security strategies and 11factors that affect the adoption of security tools. We furtherperformed an exploratory survey with 156 smart contractpractitioners from 35 countries across six continents to quan-titatively validate the security perceptions and practices thatare uncovered in our interviews. The survey respondents work Security motivators are the factors that motivate practitioners to addresssecurity; on the contrary, security deterrents are the factors that deter practi-tioners from devoting efforts to security [7]. a r X i v : . [ c s . S E ] F e b n multiple blockchain platforms, i.e., public blockchains (80),consortium blockchains (49), and private blockchains (20), andhold various job roles, i.e., development (130), testing (3),and project management (16). We investigated the followingresearch questions: RQ1. What are practitioners’ perceptions regarding smartcontract security?
85% and 69% of the survey respondents perceive the impor-tance of security and privacy in smart contracts, respectively.The security motivators include practitioners’ awareness ofimportance, workplace environment, and perceived negativeconsequences of security issues. Meanwhile, the security de-terrents include competing priorities in smart contract develop-ment and no formal process to address smart contract security.
RQ2. How does security fit into the development lifecycleof smart contracts?
This research question investigates security efforts, securitystrategies, and the adoption of security tools in smart contractdevelopment. On average, security efforts account for 29%of the overall efforts during the development process ofsmart contracts. To ensure smart contract security, practitionersdistribute efforts across different stages in the developmentlifecycle. They tend to spend significantly more effort towardssecurity at the construction and testing stages than at otherstages. In terms of security strategies and tool adoption,72% of the respondents frequently leverage more than onesecurity strategy. 58% of the respondents frequently used thecode reuse strategy. 54% of the respondents frequently adoptsecurity tools, especially the security plugins in integrateddevelopment environments.
RQ3. Do blockchain platforms influence practitioners’perceptions and practices on smart contract security?
Blockchain platforms significantly impact security percep-tions and practices of practitioners in smart contract devel-opment, including security motivators (e.g., the immutabilityof smart contracts), security deterrents (e.g., the pressure offeature delivery), the amount of security efforts across stagesin the development lifecycle (e.g., security efforts at theconstruction stage), and strategies to address smart contractsecurity (e.g., code review).Based on the findings, we discuss the disconnect betweenthe high awareness of smart contract security of practitionersand the frequent occurrence of security problems in smart con-tracts. We also provide practical lessons about code reuse, toolimplications, and proactive defense to ensure smart contractsecurity. In addition, we highlight several research avenuesacross blockchain platforms.This paper makes the following contributions: ‚ We perform a mixture of qualitative and quantitative stud-ies to investigate the security perceptions and practices insmart contract development; ‚ We provide practical implications for practitioners andoutlined future avenues of research. ‚ We provide the interview guide, questionnaire, and surveyresponses publicly accessible for future investigation by others .The remainder of the paper is structured as follows. Sec-tion II briefly reviews related work. In Section III, we describethe methodology of our study in detail. In Section IV, wepresent the results of our study. We discuss the implicationsof our results in Section V and threats to the validity of ourfindings in Section VI. Section VII draws conclusions andoutlines avenues for future work.II. R ELATED W ORK
A. Security Practices in Software Development
Practitioners work within organizations, teams, communi-ties, and cultures. Previous studies investigated the socialfactors that could impact various aspects of security practices,e.g., security tool adoption [72], [71], [68]. Organization andteam policies are a driving factor to tool adoption [10], thoughmany organizations do not encourage the adoption of securitytools. Large organizations make more use of security tools thansmall ones [72]. Existing tools fail to meet the expectations ofpractitioners by generating low-quality warning messages [10],interrupting work flow [31], [10], [60], producing excessivefalse positives [31], [10], and integrating poorly with Inte-grated Development Environments (IDEs) [31]. In this work,we investigated the factors that impact security practices insmart contract development.Security is expected to be included in developing high-quality software systems, but is rarely listed as an explicitrequirement [51]. Practitioners prioritize functional require-ments over security and focus on tasks for which outcomesare easy to measure [72], [51], [6]. Pressures from budget anddeadlines can also lead to lowering the priority of securitypractices [74]. Some organizations attempt to use penetrationtesting to motivate practitioners, but the motivation is hardto sustain without continuous support [65]. In this work,we investigated how practitioners prioritize security in smartcontract development.Building security in from the start requires a large amountof knowledge. Weir et al. found that enthusiasm about securityand motivation to learn are more likely to drive the acquisitionof security knowledge for developers than task driven [70].Alternatively, security experts could act as a roving source ofsecurity knowledge, but face challenges to convince others ofthe importance of security and examine all generated codewith limited resources [61]. Practitioners leverage variousinformation sources to gain knowledge on code security, e.g.,documentation [39], [2], and Stack Overflow [3]. Acar etal. [3] conducted an empirical study to investigate how theuse of information sources impacts code security. They foundthat developers who use Stack Overflow are more likely toproduce functional code, but less likely to write secure code.This paper investigates the involvement of security experts insmart contract development and information sources of smartcontract security. http://doi.org/10.5281/zenodo.4005112 nterview Guide Interview Transcript Semi-Structured Interviews
13 Participants
Open Card Sorting on 133 Unique Codes
Interviews
PotentialAnswers and
Statements Pilot Survey
Online Survey
156 Respondents
Survey
FindingsPilot Survey
Fig. 1. Research methodology.
In the course of software design and construction, misuseof application programming interfaces (APIs) can introducesecurity vulnerabilities [19], [21], [24]. Developers incorrectlyuse an API because they do not conduct an additional checkbut trust the API to do the right thing [44]. Acar et al.compared the usability of five Python cryptographic APIs andsuggested that documentation with examples is more helpfulthan a simple API [2]. Nadi et al. [39] performed an empiricalstudy on how developers use Java cryptography APIs. Theyfound that developers struggle with Java cryptography APIsand prefer task-based solutions. In addition, developers havedifficulty in using security-related APIs, for instance, APIs inTransport Layer Security (TLS) and Security Sockets Layer(SSL) [22], [45]. This paper investigates whether the use ofsmart contract related APIs may introduce security risks.
B. Smart Contract Security
Security vulnerabilities spread across smart contracts ofvarious blockchain platforms, e.g., Ethereum [30], [38], [56],Hyperledger Frabric [75] and EOS [52]. Security vulnerabili-ties result from multiple causes, e.g., reentrancy [56], delegate-call injection [53], and integer overflow and underflow [41].Different programming languages of smart contracts andblockchain architectures lead to different vulnerabilities [9]. Inthis work, we investigate practitioners’ awareness of securityvulnerabilities in their smart contracts.Previous studies proposed a wide range of approaches andtools for securing smart contracts, including recommendingbest practices in programming smart contracts, implementingspecific programming languages, static analysis, and runtimemonitoring. For instance, ConsenSys [13] provides extensivebest practices for Ethereum smart contract security, includingcode patterns to learn and pitfalls to avoid.
Vyper [20] and
Bamboo [16] provide language-based support to eliminatesmart contract vulnerabilities. Static analysis tools leveragesymbolic execution [14], [8], [34], [38], [43], [50], abstractinterpretation [25], [32], [59], [64], [76], formal verifica-tion [26], [5], [27], [29], [28], [49], fuzzing [37], [30], [69]and model checking [63] to identify smart contract vulnerabil-ities. DappGuard [15] and Sereum [56] monitor the runtimeexecution of a smart contract to prevent potential exploitationsof vulnerabilities. In this work, we investigate the adoption ofsecurity strategies and tools of smart contracts in practice andexplore the expectations of practitioners. III. M
ETHODOLOGY
Our research methodology followed a mixed methods ap-proach [17], as depicted in Fig. 1. The approach followsa sequential explanatory strategy, involving two phases – afirst qualitative phase of interviews, followed by a secondquantitative phase of an exploratory survey . The survey buildson the results of the interviews. Specifically, we collecteddata from two sources: (1) We interviewed 13 software prac-titioners with experience in smart contract development andderived a list of statements and potential answers for surveyquestions from the results of interviews; (2) We surveyed 156respondents with experience in smart contract development.To preserve the anonymity of participants, we anonymizedall items that constitute of Personally Identifiable Information(PII) before analyzing the data, and further considered aliasesas PII throughout our study (e.g., refer to the interviewees asP1 - P13). A. Interviews
The left part of Fig. 1 describes the process of interviews.
1) Protocol:
The first author conducted a series of face-to-face interviews with 13 software practitioners with experiencein smart contract development. Each interview took 30-45 min-utes. The interviews were semi-structured and made use of an interview guide . To develop the interview guide, we obtainedan initial set of open-ended questions through brainstormingwithin the authors of this paper, focusing on practitioners’perceptions and practices concerning smart contract security.The interview comprised three parts. In the first part, weasked some demographic questions about the experience ofthe interviewees in smart contract development. The questionscovered various aspects of experience, including programming,design, testing, and project management.In the second part, we asked open-ended questions aboutthe security awareness and practices of smart contract devel-opment. The purpose of this part was to allow the intervieweesto speak freely about their opinions and experience without theinterviewer biasing their responses.In the third part, we asked the interviewees to discussthe sources where they obtain security-related knowledge, aswell as strategies and tools that they have used for securityassurance of smart contracts in the practices. The interviews and survey were approved by the relevant institutional re-view board (IRB). Participants were instructed that we wanted their opinions;privacy and sensitive resources were not explicitly mentioned Interview guide online: http://doi.org/10.5281/zenodo.4005112 t the end of each interview, we thanked the intervieweeand briefly informed her of our next plans.
2) Participant Selection:
We recruited full-time softwarepractitioners with experience in smart contract developmentfrom blockchain companies (e.g., Hyperchain ), IT compa-nies (e.g., Alibaba) and open-source smart contract projects.Interviewees were recruited by emailing our contact in eachcompany or project, who disseminated the news of our studyto their colleagues. Volunteers would inform us if they werewilling to participate in the study with no compensation. Withthis approach, 13 volunteers with varied experience in yearscontacted us – 7 interviewees from four companies and 6interviewees from three open-source projects. In the remainderof this paper, we denote these 13 interviewees as P1 toP13. These 13 interviewees have an average of 6.5 years ofprofessional experience in software development (min: 3, max:13, median: 6.5, sd: 2.7), and an average of 2.3 years in smartcontract development (min: 1, max: 5, median: 2, sd: 1.1).Table I summarizes the number of interviewees who perceivedthemselves as having “extensive” experience (in comparisonto “none” and “some” experience) in a particular role.
3) Data Analysis:
We conducted a thematic analysis [18] toprocess the recorded interviews by following the steps below:
Transcribing and Coding.
After the last interview wascompleted, we transcribed the recordings of the interviews,and developed a thorough understanding by reviewing thetranscripts. The first author read the transcripts and coded theinterviews using NVivo qualitative analysis software [1].To ensure the quality of codes, the second author verifiedinitial codes created by the first author and provided sugges-tions for improvement. After incorporating these suggestions,we generated a total of 427 cards that contain the codes - 30to 41 cards for each coded interview. After merging the codeswith the same words or meanings, we have a total of 133unique codes.
Open Card Sorting.
Two of the authors then separately ana-lyzed the codes and sorted the generated cards into potentialthemes for thematic similarity (as illustrated in LaToza etal.’s study [35]). The themes that emerged during the sortingwere not chosen beforehand. We then use the Cohen’s Kappameasure [12] to examine the agreement between the twolabelers. The overall Kappa value between the two labelersis 0.76, which indicates substantial agreement between the la-belers. After completing the labeling process, the two labelersdiscussed their disagreements to reach a common decision.To reduce bias from the two authors sorting the cards to forminitial themes, they both reviewed and agreed on the final set ofthemes. Eventually, we derived 6 competing priorities, a list of11 security motivators and 9 security deterrents, 5 sources ofsecurity knowledge, and 11 security strategies, and 11 factorsthat affect the adoption of security tools.
B. Survey
The right part of Fig. 1 describes the process of our onlinesurvey. UMBER OF INTERVIEWEES WITH “ EXTENSIVE ” EXPERIENCE IN APARTICULAR ROLE . Role Smart Contract non-Smart-Contract
Programming 10 12Design 8 6Management 3 4Testing 3 3
1) Protocol:
We conducted an IRB-approved anonymousonline survey with professional smart contract practitioners.The survey aims to validate and quantify the observations fromour interviews. We followed Kitchenham and Pfleeger’s guide-lines for personal opinion surveys [33] and used an anonymoussurvey to increase response rates [66]. A respondent has theoption to specify that she prefers not to answer or does notunderstand the description of a particular question. We includethis option to reduce the possibility of respondents providingarbitrary answers.
Recruitment of Respondents.
To recruit respondents fromthe population of smart contract practitioners, we spread thesurvey to a broad range of companies from various loca-tions worldwide. No identifying information was required orgathered from our respondents. To get a sufficient number ofrespondents from diverse backgrounds, we followed a multi-pronged strategy to recruit respondents: ‚ We contacted professionals from blockchain companiesand IT companies that launch blockchain projects aroundthe world and asked their help to disseminate our surveywithin their organizations. Specifically, we sent emailsto our contacts in Alibaba, Baidu, Hengtian, Hyperchain,IBM, Morgan Stanley, and other companies, encourag-ing them to disseminate our survey to some of theircolleagues who have experience in smart contract devel-opment. By following this strategy, we aimed to recruitrespondents working with smart contracts in the industryfrom diverse organizations. ‚ We sent an email with a link to the survey to 1,986practitioners who contributed to 12 blockchain repos-itories that support smart contracts (e.g., ethereum/go-ethereum , EOSIO/eos and hyperledger/fabric ) and 580smart contract repositories (e.g., ethereum/solidity and
EOSIO/eosio.contracts ) hosted on GitHub and solicitedtheir participation. We aimed to recruit open-source prac-titioners who have smart contract experience in additionto professionals working in the industry.Out of these emails, eight emails received automaticreplies notifying us of the absence of the receiver; twoemails indicated the receivers left the original organiza-tions; four receivers replied that they only have experi-ence in blockchain but not smart contract development.
2) Survey Design:
The survey includes different types ofquestions, e.g., multiple-choice and free-text answer questions.The potential answers and statements of multiple-choice ques-tions were derived from the results of our interviews. For thesequestions, we include an “
I don’t know ” option in case somestatements are not applicable to the experience of respondents,r respondents had a poor understanding of the statements.The survey consists of four sections, grouping questionsby topic to minimize the cognitive load on participants andallow them to consider the topic more deeply [36]. Specifically,the following four sections have been captured in the survey(the complete questionnaire is available online as supplementalmaterial ): Demographics.
We collected demographic information aboutthe respondents to allow us to (1) filter respondents whomay not understand our survey (i.e., respondents without anyexperience in smart contracts), (2) breakdown the results bygroups (e.g., public, consortium, and private blockchains).Specifically, we asked two questions: ‚ Do you have experience with smart contracts? ‚ What best describes the primary blockchain platform that you currently work on ?In terms the second question, we provided four options forprimary blockchain platforms, including (1) public blockchain ,(2) consortium blockchain , (3) private blockchain , and (4) other .Based on the selections of respondents, we could excludeinvalid responses and divide the survey respondents into threegroups. To focus the respondents’ attention on a particularblockchain platform in the survey, they were explicitly askedto answer each following question with respect to their expe-rience with the primary blockchain platform they specified.We received a total of 203 responses, and further excluded46 responses made by respondents who claimed that they donot have experience in smart contract development. We alsoexcluded one response made by a respondent who describedher job role as sales. In the end, we had a set of 156 validresponses. The 156 respondents reside in 35 countries acrosssix continents as shown in Fig. 2. The top two countries inwhich the respondents reside are China (61) and the UnitedStates (16). The respondents have an average of 6.3 yearsof professional experience (min: 0.5, max: 40, median: 4,sd: 6.9), with an average of 2 years of experience in smartcontract development (min: 0.1, max: 6, median: 2, sd: 1.4).Our survey respondents are distributed across different demo-graphic groups (job roles and primary blockchain platforms)as shown in Fig. 3. Seven respondents who selected
Other as their primary blockchain platforms and explained thatthey simultaneously work on more than one blockchain. Weexcluded the responses of the seven respondents from anycomparisons between groups of different blockchains.
Perceptions on Smart Contract Security.
This section in-vestigates practitioners’ perceptions of smart contract security,specifically, the importance of security, awareness of securityproblems, as well as the motivators and deterrents to smartcontract security.
Security Practices in Smart Contract Development.
Thissection focused on security practices in smart contracts, in-cluding practitioners’ efforts towards security, their strategiesfor achieving security, and tools for securing smart contracts. Questionnaire Online: http://doi.org/10.5281/zenodo.4005112
Fig. 2. Countries in which survey respondents reside. The darker the color is,the more respondents reside in that country. The legend indicates the numberof respondents. 3 U L Y D W H % O F R F N F K D L Q &