SkillCheck: An Incentive-based Certification System using Blockchains
SSkillCheck : An Incentive-based CertificationSystem using
Blockchains
Jay Gupta and Swaprava Nath
Indian Institute of Technology Kanpur { jaygpt, swaprava } @iitk.ac.in Abstract —Skill verification is a central problem in workforcehiring. Companies and academia often face the difficulty ofascertaining the skills of an applicant since the certifications ofthe skills claimed by a candidate are generally not immediatelyverifiable and costly to test. Blockchains have been proposed inthe literature for skill verification and tamper-proof informationstorage in a decentralized manner. However, most of theseapproaches deal with storing the certificates issued by traditionaluniversities on the blockchain. Among the few techniques thatconsider the certification procedure itself, questions like (a) scal-ability with limited staff, (b) uniformity of grades over multipleevaluators, or (c) honest effort extraction from the evaluators areusually not addressed. We propose a blockchain-based platformnamed SkillCheck, which considers the questions above, andensure several desirable properties. The platform incentivizeseffort in grading via payments with tokens which it generatesfrom the payments of the users of the platform, e.g., the recruitersand test takers. We provide a detailed description of the design ofthe platform along with the provable properties of the algorithm.
Index Terms —skill verification, blockchains, strategic graders,honest effort
I. I
NTRODUCTION
Blockchain technology has seen an unprecedented growthafter being introduced in 2008. First used as a peer-to-peerledger for registering the transactions, it has seen its owntechnological development as well as its adoption in variousapplications. It is desirable in many applications primarilybecause it eliminates any third-party intermediary and allowsusers to make their transactions directly. In a blockchain, eachnode in a decentralized network of peer nodes (1) holds areplica of a common ledger, (2) writes an entry to its ownledger when it has agreement from the other nodes in thenetwork, (3) broadcasts any transaction made by its user tothe other nodes in the network, and (4) checks, on a regularbasis, that the ledger it holds is identical to the ones acrossthe network [1].On another hand, the rapid development of the online edu-cation, powered by the massive open online courses (MOOC),has also seen an unprecedented growth. In the process of thisgrowth, it needed some of the properties that are ubiquitousin the blockchains, most notably, (i) the absence of a centralauthority and (ii) verifiable certifications. Skill certificationtoday demands need-based skills to be certified which is alittle difficult to come by in a standard university system. Forinstance, a course on android app development may not find
This work is supported by the IIT Kanpur fund number cs/2017198. a place in a university curricula, while a student skilled inthat may be indispensable for certain organizations. Yet, theorganization has to hire the student uncertified or has to testthe skill by itself, which is expensive.To bridge this gap and to provide skill certification ondemand, the need for blockchains in education has emerged[15]. This need is also reflected in the European commissionpolicy report [6], which agrees that the blockchain applicationsin education is still at infancy, and recommends thatTo ensure development of open blockchain imple-mentations we recommend that the EU in collabora-tion with Member States consider creating and pro-moting a label for ‘open’ educational records, whichenshrines the principles of recipient ownership, ven-dor independence, and decentralised verification.The literature on application of blockchains in education isheavy with the management of certificates issued by thestandard authorities. A survey by Alammary et al. [1] showsthat almost 41% of the systematically surveyed articles dealtwith this issue. However, a major impediment to the skillcertification is to get the right instructor and evaluators thatcan certify the skills of candidates. This is a challengingtask since there is little work in this domain that study the quality of the certificates or on ensuring the best effort of theevaluators. In this paper, we take an incentive -based approachfor the different players in the education blockchain networkand investigate what guarantees are possible to provide in the quality of certifications . A. Our contributions
We propose a blockchain-based education platform called
SkillCheck , which handles large classes with limited num-ber of teaching staff, provides uniformity of grades, andensures honest effort from the evaluators. This platform canbe implemented inside a university or an organization, orcan be implemented by onboarding multiple universities ororganizations on a single large blockchain. It allows any userto offer a course according to the need-based skill certificationsof the other users. Hence, it has the flexibility of designingand certifying non-traditional courses in an asynchronous manner, i.e., the beginning or end does not need to alignwith any semester/quarter. The instructor can choose a set ofevaluators in the blockchain network who are skilled in thattopic, e.g., individuals who have been certified in the samesystem earlier.
SkillCheck uses crypo-tokens for transferring a r X i v : . [ c s . CR ] M a r redit for good evaluation to the evaluators (Section II). Theunderlying mechanism of SkillCheck ensures that the rewardsof evaluation is designed such that the individual biases ofevaluators have no effect in the final score received by thecandidate. Therefore, it is ensuring uniform grading of thepapers. Also, the mechanism ensures that if an evaluatorexerts effort to reduce her noise of grading, her pay-offincreases (Theorem 1). We provide layer-wise divisions ofthe functionalities of the platform (Section III) and a detaileddescription of the implementation of the platform (Section IV)so that it is reproducible.The complete details are available in the full version of thepaper [8].
B. Related work
The nascent research area of blockchains in education canbe classified in many strands. Here we will discuss about twolargest strands. The first strand is focused towards certificatesmanagement [5, 7, 9, 11, 13, 14], which is a secure decen-tralized way to store the traditional university certificates. Ourwork is closest to the second strand of literature that deals withthe management of skill or competencies [3, 4, 9, 10, 14, 15,17]. These works focus on generating the competencies basedon the evaluations of the evaluators. However, the evaluatorsare assumed to be honest graders and there is no normalizationof the scores given by different evaluators, if used. Alammaryet al. [1] provide a nice classification of the literature based ontheir category of applications. The literature, however, is thinon the algorithmic guarantees on competencies of the evalu-ators. Our approach fills this important gap with a platformwith provable algorithmic guarantees for skill certification.II. D
ESIGN W ORKFLOW
The basic platform of skill certification in this paper is ablockchain with users who can have exactly one of the four roles at any given instant: (a) the instructor, who designsand conducts an exam, sets the guidelines of grading, andalso partially grades answerscripts, (b) the evaluators, who areindividuals with sufficient expertise in the topics of the exam– they can be past creditors of the course, (c) the candidatesseeking certification, (d) the viewers of the certificates, e.g.,the recruiters who are looking for individuals of a certainskill. We assume that the number of candidates is quite largefor the instructor to check. Therefore, he needs evaluators,e.g., the teaching assistants, who help him grading the papers.However, the evaluators may spend little or no effort inthe grading carefully, which can lead to a bad quality ofcertification. So, in such a context of decentralized evaluationand certification, we need certain desirable properties of anyplatform, given as follows. • Scalability : the platform should handle a large number ofcandidates. • Uniformity of scores : if the scores are given by differentevaluators having different degrees of noise, it is necessarythat the final grade is normalized in some way to ensureuniformity of the grades. • Honest effort extraction : evaluators must be incentivized tomaximize their effort in grading the papers.The certification obtained via this platform will always bestored in a blockchain. Therefore, the properties like veri-fiability, absence of a central repository of certificates, andauthenticity of the records are inherited by default, and arenot mentioned as separate desiderata.To ensure the desirable properties, our approach is to exploitthe fact that a grading mechanism (a) can have some instructor-graded answer-scripts (which we call probes ) to measure the quality of the evaluators, and (b) allow students to raise a regrading request for incorrect grades which can be correctedby the instructor. Hence, eventually all ‘true’ grades of thepapers, where the ‘given’ scores are below the ‘true’ scores,will be obtained, and this information will be used to deter theevaluators from deliberately under-performing. Our approachis inspired by the
TRUPEQA mechanism [2], but modifiedfor our setting.In the following, we outline the functional structures of ourproposed platform
SkillCheck . We will discuss the model ofthe evaluation process and the provable properties it satisfiesin subsequent sections.
SkillCheck token.
Crypto tokens are well-known holdings inthe crypto space and are usually called cryptocurrency in theblockchain parlance. It is a standard currency which is usedto make and receive payments on a blockchain network. Incontrast to fiat currency, crypto tokens are a special kind ofvirtual representation of a particular asset or utility, that resideon the blockchain network.In our setting, all four types of users of
SkillCheck will haveindividual wallets attached to their account, which contain theamount
SkillCheck tokens that user (represented by a nodein the network) hold, and individual portfolios showing theirskill score and evaluation scores (relevant for the candidatesand evaluators) that constitute their certificates. The skill scoreshows the knowledge of the candidates on a subject andevaluation score reflects the evaluator’s skill in grading ananswersheet. To begin with, the wallets can be recharged byconverting fiat currency into the tokens. However, we proposeto waive the transaction costs of the evaluators, since they arethe workforce for the
SkillCheck platform.The examiners and candidates pay
SkillCheck tokens asa fee to conduct any examination and to get certified respec-tively. The evaluators also need to pay a small threshold tokento enroll themselves as an evaluator. The total collected token(from initial token charged to the examiner and small tokenpaid by the evaluators) will be distributed to evaluators basedon the evaluation score that evaluator will earn by grading thepapers. The steps of the workflow of
SkillCheck is given asfollows (see Figure 1).In step 1, an examiner submits an exam on blockchainnetwork by calling a public function to create a new copy nstructor
SkillCheck
Back-endAlgorithm forCalculating score Evaluation by evaluatorsPooledResponsesEXAMINATIONCandidates go to examcenter and give exam EXAMINATIONPublicly available exam created byinstructor on block-chain by creatingexam SCDistribution Algorithm
Candidate and evaluatorenroll themselves inSkillCheck Addition ofprobe papersComputation of scores Display oftentativetest score Transfer of scoreto candidateRegradingrequest ifneeded Finalizing the scoreDistributionof tokenstoevaluators Recruiters canaccess portfolio
ApplicationLayerContractLayer NetworkLayerIncentiveLayerConsensusLayerData and PhysicalLayer are inSkillCheck Portal
Evaluator
Fig. 1. The workflow diagram in
SkillCheck . of a Smart Contract (SC). Placing an examination on theblockchain costs him some fixed number of tokens. After-wards, the paid token for the submission is locked withinthe SC. Users’ wallets associated with the blockchain networkcontains the portfolio in which users will have skill and evalua-tion scores, along with SkillCheck token which can be earnedby evaluating papers or purchasing from the system. Oncethe question paper of the exam is on the blockchain network,eligible candidates can enroll themselves in examination bycalling the public function of the SC. The students have to goto the local examination centre for giving the exam.In step 2, candidates and evaluators enroll themselves usingtheir public keys.In step 3, the examination will be conducted at a testcenter to prevent identity theft. Every candidate gets a fixednumber of maximum attempts to get themselves certified. Thecandidates need to solve the paper and submit the responsesby calling the private function of the SC, which ensures theprevention of data leakage threat. The SC will contain allpooled answers of candidates.In step 4, the examiner checks a constant number ofanswerscripts that constitutes the probe papers. This is neededas our algorithm (explained in Section III-B) needs the probes A smart contract, also known as a cryptocontract , is a computer programthat directly controls the transfer of digital currencies or assets between partiesunder certain conditions. An algorithm can be encoded into a smart contractwhich automates the payment process. to estimate the accuracies of the evaluators. The scores of theprobe papers are assumed to be benchmarks as it is graded bythe instructor. A distribution strategy is implemented in the SCthat distribute sets of papers to all evaluators. Every evaluationgets an equal mix of anonymous probe and non-probe papers.The evaluators check the papers assigned to them and submittheir scores for the assigned papers. A small gas price forthis evaluation transaction is charged to the evaluators. Thisprice is a constant and is later reimbursed by the platform. Theevaluators need to call the SC function and submit evaluatedmarks to contract.In step 5, our algorithm computes the skill scores of thecandidates participating in the exam. The scores are sharedwith the candidates.In step 6, the candidates can raise a regrading request ifthey feel the scores are unfair. The paper is then checked bythe instructor to find the “true” score. Our algorithm keepsa penalty for frivolous regrading requests to entertain onlyreasonable regrading requests. Due to the properties of ouralgorithm, which we prove later, the number of regradingrequests are expected to be small.In step 7, based on the evaluations of the instructor onthe papers that went through the regrading process and thepapers for which there were not any regrading request, thefinal skill scores of the candidates and the evaluation scoresof the evaluators are updated in the SC. For the papers thatid not raise any regrading request, the final score is the scoregiven by the algorithm in step 5.Step 8 computes an appropriate distribution of collected to-ken from evaluators and examiner and the SC will transfer thecollected tokens to evaluators. The skill scores are registeredin the profile of the candidates via the SC.In step 9, the viewers, e.g., the recruiters, can view the skilland evaluation scores of the users by paying tokens.III. L
AYER - WISE D ESIGN OF
SkillCheck
Since the development of the platform
SkillCheck needsall functionalities of a blockchain environment to be imple-mented, it is often easier to break them down into multiplelayers of specific functionalities. We follow the layered struc-ture given by Yuan and Wang [16]. In this section, we providea detailed view of the first three layers, i.e., the application,contract, and incentive layers. For the rest of the layers, i.e.,the consensus, network, data, and physical layers, we use thestandard implementation of the Ethereum framework and itsdefault configuration. Figure 2 shows the layered architectureof our proposed platform.
NetworkLayer IncentiveLayerContractLayerConsensusLayerDataLayer
LAYERS OF BLOCKCHAINMODEL
ApplicationLayerPhysicalLayer
Fig. 2. Layered architecture of the
SkillCheck platform.
A. Application Layer
This layer deals with the application of skill verificationof users, authentication of certificates issued by the variousinstitutes using blockchain networking, and stores the tokensin the user wallets. In the
SkillCheck platform, this layer dealswith the development of the user interface that shows theirprofile with (a) the certificates issued by the platform, (b) thecoins in the wallet, and (c) the roles that they can take–e.g.,instructor, candidate, evaluator, or recruiter.
B. Contract Layer
This layer mainly consists of various scripts for func-tionalities, which serve as a necessary means to communi-cate with the whole network, including blocks stored in theblockchain. This layer also implements the central algorithmof
SkillCheck . The algorithm and functionalities are encoded into a smart contract, which is a group of self-authenticating,self-administering, and self-driving quick response rule sys-tem that is stored on the blockchain. To present the centralalgorithm of
SkillCheck , we develop the basic notation.The notation and the desiderata are similar to the propertiesmentioned in [2]. However, unlike [2], here the evaluation ofthe papers are done by evaluators who are different from thetest takers.
1) Setup:
Each candidate interested in getting certifiedhas written an exam that needs to be evaluated. Let N = { , . . . , n } represent the set of the candidates and M = { , . . . , m } denote the set of evaluators. We use i as the indexfor an evaluator and j as the index for a paper (of a candidate).Out of the n total papers, (cid:96) ( << n ) papers are evaluated bythe instructor and these papers are called the probe papers.We assume that through this grading process, the true grades of these probe papers are known to the designer (e.g., theinstructor of the course). The scores belong to a closed interval S ⊂ R .An evaluation mechanism E is the tuple (cid:104) G, r , t (cid:105) , where • G is an assignment function G : M (cid:55)→ N that maps papersto evaluators. The set of papers graded by i is G − ( i ) (cid:44) { j ∈ N : i ∈ G ( j ) } . • r : × i ∈ N S G ( i ) → S n , where the j th component r j ( · ) is thefunction assigning the skill score of candidate j based onthe scores reported by the assigned evaluators. • t : × i ∈ N S G ( i ) → S n , where t i ( · ) is the function that yieldsthe evaluation score to evaluator i .The set P i ⊂ G − ( i ) and N P i = G − ( i ) \ P i denotes respec-tively the probe and non-probe papers assigned to i . We definethe co-evaluators of individual i as CG i = ∪ j ∈ NP i G ( j ) \ { i } ,i.e., the set of evaluators who grade at least one common non-probe paper with i .
2) Model of true and reported scores:
We assume thatthe generation of the true scores and the error model ofthe evaluators is given by the PG model of evaluator biasand reliability as described in Piech et al. [12], described asfollows. • The true score y j for paper j is distributed as N ( µ, /γ ) ,for all j ∈ N . The parameters of this distribution are learnedfrom the historical data of examinations. • The reported score of paper j by evaluator i , given by ˜ y ( i ) j ,is distributed as N ( y j + b i , /τ i ) , where b i and τ i are calledthe bias and reliability of i respectively.The bias term may be interpreted as either strategic (manip-ulating grades in a particular direction consciously) or non-strategic (subconsciously generous or strict evaluators). Weassume that an evaluator can choose her reliability in thegrading process. Intuitively, bias resembles to what extentthe evaluator is shifting the mean of the score and reliabilitydenotes how much effort she is putting in to find the scoreaccurately. A crucial assumption here is that an evaluatorgrades all papers in an exam (particularly, the probes and non-probes) with the same bias and reliability. We believe thatthis assumption is not too restrictive in an environment wherethe evaluators cannot distinguish the probes from the non-robes (as they are given a mix of them without mentioningthe distinction). We use the shorthand θ i := ( b i , τ i ) ∈ R × R (cid:62) ,where b i (potentially) and τ i are evaluator i ’s strategic choices.
3) Description of our mechanism (cid:104) G ∗ , r ∗ , t ∗ (cid:105) : The compo-nents of our evaluation mechanism (cid:104) G ∗ , r ∗ , t ∗ (cid:105) are as follows.Let K be an even predetermined number of papers eachevaluator grades. This number is an exogenous design choiceof the platform. a) Paper assignment rule G ∗ ( · ) : We impose (a) G ∗ ( j ) (cid:54) = ∅ , ∀ j ∈ N , (b) | P i | = K/ , ∀ i ∈ M , and (c) | N P i | = K/ , ∀ i ∈ M , i.e., every paper is graded by at least oneevaluator, and every evaluator grades K/ probes and K/ non-probe papers. The evaluators know the numbers of probeand non-probe papers assigned to them, but cannot tell themapart. b) Skill and evaluation scores: Once the evaluatorsscores are collected, our mechanism statistically estimates theerror parameters ˆ θ i : S | P i |×| P i | (cid:55)→ R × R (cid:62) of each evaluator i , from the comparison between reported grades ˜ y ( i ) j s andtrue grades y j s on the probe papers P i . In our particularcase, this is given by ˆ θ i = (ˆ b i , ˆ τ i ) , and ˆ b i = (cid:80) j ∈ Pi ˜ y ( i ) j − y j | P i | and ˆ τ i = | P i |− (cid:80) j ∈ Pi (˜ y ( i ) j − ( y j +ˆ b i )) . Below we describe how theestimated parameters are used in generating the skill scores ofthe candidates and the evaluation scores to evaluators.D EFINITION
We define aspecific score computing rule that would be used in aggre-gating the evaluator-reported-scores into a final score on eachexam. We also define a social welfare criteria that would beused to describe the evaluation scores. • The score computing function r ∗ = ( r ∗ j : j ∈ N ) ofan evaluation mechanism E is inverse standard-deviationweighted de-biased mean (ISWDM) if it assigns theinstructor-verified grade on every probe paper. For everynon-probe paper j , it assigns r ∗ j ( ˜y G ( j ) j , ˆ θ G ( j ) ) = √ γµ + (cid:80) i ∈ G ( j ) √ ˆ τ i (˜ y ( i ) j − ˆ b i ) √ γ + (cid:80) i ∈ G ( j ) √ ˆ τ i . where ˜ y ( i ) j is the score assigned by the i th evaluator and (ˆ b i , ˆ τ i ) are the parameters we estimated for her. • The scores r ∗ are released to the candidates, and they canraise a regrading request if they feel the score is unfair. Theinstructor checks the paper in such a case and decides thefinal score y j , which we assume to be the true score ofthe paper. If candidate j do not raise a regrading request,the mechanism assumes the true score to be r ∗ j . Here wecan explicitly say that the total effort(in checking all papershimself) made by the instructor, has been reduced by asubstantial amount. To avoid the case where every candidate may raise a regrading request(because it does not hurt), we can impose penalties for frivolous requests.This threat mechanism works well in practice and has been part of variousonline grading tools. • The social welfare at a score r ∗ j for paper j when the truescore is y j is denoted by W ∗ j ( ˜y G ( j ) j , ˆ θ G ( j ) , y j ) = R ( r ∗ j ( ˜y G ( j ) j , ˆ θ G ( j ) ) , y j ) . where ˜y G ( j ) j is the vector of evaluated scores reportedon paper j , and ˆ θ G ( j ) is the vector of estimated error-parameters for the relevant evaluators G ( j ) .We assume that R ( x, x ) = 0 ≥ R ( x, y ) = R ( y, x ) for all x, y ∈ S , and R ( x , y ) (cid:62) R ( x , y ) if | x − y | < | x − y | .One example of such a function would be R ( x, y ) = − ( x − y ) , which calculates the squared error in assigned scores. • The social welfare at a score r ∗ j for paper j withoutevaluator i when the true score is y j is denoted by W ( − i ) ∗ j = W ∗ j ( ˜y G ( j ) \{ i } j , ˆ θ G ( j ) \{ i } , y j ) where W ∗ j ( · ) isdefined as before. • The evaluation score of evaluator i for grading paper j ∈ N P i is given by t j, ∗ i = α ( W ∗ j − W ( − i ) ∗ j ) , where α > is a constant chosen at the designer’s discretion. The totaltransfer to evaluator i is therefore t ∗ i = (cid:80) j ∈ NP i t j, ∗ i .The parameters γ, µ, b i , and τ i are from the PG model ofPiech et al. [12] as defined at the end of Section III-B2.We will use the shorthands W ∗ j and W ( − i ) ∗ j for the abovetwo expressions when the arguments of such functions areclear from the context.
4) Incentives and design desiderata:
In this section, wedescribe the preferences of the population participating in themechanism. A candidate j can get only skill scores, r ∗ j or y j . Her strategy to improve this score is via putting moreeffort in getting the answers correct and raise correct regradingrequests. Hence there is no strategic aspect in a candidatestrategy. An evaluator i , however, can have a conscious orunconscious bias b i and can choose the amount of effort,represented via the reliability τ i , that she can exert to gradethe papers assigned to her. The pay-off (we use pay-off andutility interchangeably in this paper) of the evaluator is theevaluation score t ∗ i . Depending on how t ∗ i is related to the biasand reliabilities, evaluator i can choose those two parametersto maximize her incentive. This is where the incentive designquestion is important. Denote the utility of evaluator i by u ( G ∗ , r ∗ , t ∗ ) i ( b i , τ i , b − i , τ − i ) = t ∗ i . The objective of the
SkillCheck platform is to ensure thatthe evaluation is done with uniformity of skill scores andby extracting honest effort from the evaluators. Hence thefollowing properties are desirable in such a setting.A few different uncertainties are resolved after the evaluator i chooses her decision variables ( b i , τ i ) , and before r ∗ and t ∗ are computed. These are: (1) the scores finally reported bythis evaluator i , which on any paper j , follow the distribution f (˜ y ( i ) j | y j ) ∼ N ( y j + b (cid:48) i , /τ i ) , (2) the decision variables ( b k , τ k ) chosen by every co-evaluator k , (3) the true score y j on paper j , (4) the scores finally reported by a co-evaluator k , which on any paper j follow the distribution f (˜ y ( k ) j | y j ) ∼ N ( y j + b k , /τ k ) , conditional on the decision The notation ‘ − i ’ denotes all the evaluators except i . ariables ( b k , τ k ) . The properties defined below consider theevaluator i ’s expected utility from the choice of strategiesshe makes, where the expectation is only being taken withrespect to the distribution of i ’s grade-evaluation process f (˜ y ( i ) j | y j ) ∼ N ( y j + b i , /τ i ) , ∀ j ∈ G − ( i ) , and they holdfor any realization of the other uncertainties (2) to (4).D EFINITION
An eval-uation mechanism E = (cid:104) G, r , t (cid:105) is ex-post bias insensitive ,if the expected utility of every evaluator i ∈ M , whereexpectation is taken with respect to the distribution of i ’sgrade-evaluation process, is independent of the bias b i . Thisholds irrespective of the biases and reliabilities chosen bythe other evaluators k (cid:54) = i , and the realized value of thetrue score y j and reported scores of the different evaluators.Mathematically, for all { ˜ y ( k ) , b k , τ k } k (cid:54) = i E ˜ y ( i ) G − i ) | y G − i ) u ( G, r , t ) i ( b i , τ i , b − i , τ − i )= E ˜ y ( i ) G − i ) | y G − i ) u ( G, r , t ) i ( b (cid:48) i , τ i , b − i , τ − i ) . (1)EPBI ensures that the expected utility of a evaluator is in-sensitive to her bias for (a) every realization of the randomvariables governing the true and reported scores, (b) the bias,reliabilities chosen by the other evaluators, and (c) her ownreliability. The next property discusses the dependency withreliability.D EFINITION
An evaluation mechanism E = (cid:104) G, r , t (cid:105) is ex-post reliabilitymonotone if for every evaluator i , the utility is monotonicallynon-decreasing with her reliability irrespective of the biasesand reliabilities chosen by the other evaluators and the re-alizations of the true scores and the scores reported by thedifferent evaluators. Mathematically, for all { ˜ y ( k ) , b k , τ k } k (cid:54) = i E ˜ y ( i ) G − i ) | y G − i ) u ( G, r , t ) i ( b i , τ i , b − i , τ − i ) (cid:62) E ˜ y ( i ) G − i ) | y G − i ) u ( G, r , t ) i ( b i , τ (cid:48) i , b − i , τ − i ) , ∀ τ i > τ (cid:48) i . (2)Note that these properties are stronger than a similar definitionin dominant strategies . Since there are random variables likethe true scores y and the reported scores ˜y respectively, andthey can change the utility expression, a dominant strategydefinition will only require the (in)equalities to be satisfiedafter taking interim expectation over some relevant distributionover those variables. However, the ex-post properties requirethem to be satisfied for every realization of these randomvariables.Our central result on the proposed evaluation mechanism isas follows. We defer the proof to the appendix for a cleanerpresentation.T HEOREM
Mechanism ( G ∗ , r ∗ , t ∗ ) is EPBI and EPRM.This result ensures that any rational evaluator should notput any bias to the evaluations and work at their highestreliability. The ( G ∗ , r ∗ , t ∗ ) mechanism forms the backboneof SkillCheck . Hence the platform satisfies the properties asthat of this mechanism. In the next layer, we discuss how theincentives of the evaluators are implemented.
C. Incentive Layer
Incentive layer is designed to incorporate the rewards tothe participants (the evaluators in our case) to satisfy theobjectives of the blockchain network. In
SkillCheck , thedesirable properties are EPBI and EPRM. These are obtainedthrough the award of the evaluation score t i to evaluator i ∈ M . Our platform transfers t i amount of tokens to evaluator i , which they can convert to fiat currency. However, we needto ensure that the platform does not run into an overall lossafter such transfers. The platform earns the tokens from thepayments of (a) the candidates who are willing to get certified,(b) the recruiters, who wants to view the certificates of thecandidates, and (c) the instructor, who needs to pay to offer acourse on SkillCheck . Let the collective amount from all thetokens received in these way is K net . SkillCheck ensures thatthe expected sum of transfers, i.e., E ( (cid:80) i ∈ M t i ) , is below K net ,where the expectation is taken over the prior distribution of theskill scores, evaluators’ error distributions, and the observedscores of the evaluators. The distributions can be obtainedfrom the historical data of the course. We call this property expected budget balance , which can be achieved by setting theparameter α in the expression of t ji based on the K net and theexpected value of the sum of the evaluation scores.For a practical implementation, the estimate of α may beset conservatively, i.e., that ensures that the K net is less likelyto get exhausted. However, the choice of α for a specific examneeds to be derived empirically using the available historicalinformation of that exam.The rest of the layers are implemented according to anystandard protocol of blockchain platforms used in practice. Wefollow the implementation of the Ethereum and the details aregiven in the next section.IV. B LOCKCHAIN I MPLEMENTATION
To implement
SkillCheck , we use the Rinkeby Test net-work provided by the Ethereum Foundation. In the currentimplementation, we have used the Ethereum private networkto establish a blockchain system. We have used Solidity toperform the primary functionality of a smart contract (SC).Every action in the platform
SkillCheck , e.g., checking theprobe papers by the instructor, evaluations of the answerscriptsby the evaluators, raising regrading requests by the candidates,and assignment of final skill and evaluation scores by theinstructor, is considered a transaction, which is executed viaa cryptographically signed contract that runs on the net-work. The codes for full implementation can be found at: https://github.com/jaygpt/Skillcheck .The implementation of
SkillCheck is divided into three main segments: (1) the front-end that deals with the userinterface, (2) the smart contracts where the algorithms areimplemented, and (3) the back-end which connects
SkillCheck to the Ethereum blockchain. These components are explainedas follows. nrollment address[] public students;mapping(address=>string) names;mapping(address => address) wallet;mapping(address => bool) doyouhavewallet;+ function createwallet(string) public{}+ function totalwallet() public view returns(address[])+ function mywallet() public view returns(string)
Wallet address public student;uint score;uint testevaluatingscore;address admin;+ function Setadmin(addressexaminer) public+ function Wallet(address )public+ function setscoretest(uint ) public+ function setscoreeval(....) public+ function presentscore() public viewreturns+ function myaddress() public viewreturns(address)
Exam Factory address[] myaddress;mapping(address => string)Instructions;+ function creatorTest(string) public+ function creatorTest(string ) public+ function totalwallet() public viewreturns (address[])+ function totalinstruction(address)public view returns(string)
Exam mapping(address => bool) public attende;struct Marksbyme{address, int[]};mapping(address => Marksbyme[]) grading;address[] studentswallet;address examiner;string[] public Questionset;struct questionanswer{string, string, int}mapping(address => questionanswer[]) public exam;+ function addProbe(address, uint , int) public restricted+ function Test(address creator) public{}+ function addQuestion(string question) public restricted{}+ function enrollintest(address walletaddress) public{}+ function starteval(address walletaddress) public{}+ function answermytest(address walletaddress,uint index,string answer) public{}+ function becomegrader(address walletaddress, address roll) public{}+ function givemarks(address roll, address Mywallet, int marks,uint index) restricted2 public {}+ class examinerAuthority(address , address , int ,uint ) restricted2 private {}
Smart contract tocreate new testSC needs to execute atenrollment
COLLEGE (SMARTCONTRACT) TEST (SMART CONTRACT)
Fig. 3. Overview of the smart contracts of
SkillCheck . Front End Decentralized ApplicationBack end(Network)
Web3Web3 responseJSON RPC Protocol
Fig. 4. The architecture of
SkillCheck . A. Back-end
Like most of the Ethereum’s decentralized application(DApp),
SkillCheck also has Node.js as its backend frame-work. We have used a combination of Node.js and MongoDBto maintain the database and to manage the connection be-tween the database and the blockchain interactions and todisplay using our frontend and render methods. However,
SkillCheck uses web3.js JavaScript library which allows theclient-side to interact with the blockchain including a localor a remote Ethereum node, using a HTTP or inter-processcommunication (IPC) connection. This system is more securethan a general web application as the user does not need togive control of her private keys to the service provider, whichis not possible in a web application. Moreover, the interactionwith the portfolio and the wallet, where the tokens are stored,is administered only by the client-side.
B. Front-end
As a blockchain works primarily on the client-side, it isessential to have a proper interface for interacting with theEthereum blockchain directly. Any DApp is similar to a webpplication in multiple ways. Using various JavaScript webtechnologies, an easy-to-use interface supports
SkillCheck
Rinkeby blockchain/off-chain interaction. MetaMask, that al-lows us to run Ethereum DApps in the browser withoutrunning a full Ethereum node, is used for submitting responsesto an SC, to create a response sheet to the SC by the evaluatorand other user-friendly interfaces. Truffle framework is usedto create a new wallet and Ganache-CLI is used for Ethereumblockchain simulation. ReactJS (a JavaScript library) is usedfor building user interfaces especially for the single pageapplications such as the enrolment page.
C. Smart Contract and Ethereum
An abstract description of the
SkillCheck
SCs is given inFigure 3. Each block in the figure depicts two segments: thefirst shows the objects needed for the SC, and the secondpart shows the functions used in the SC. Using the standardweb3.js JavaScript library,
SkillCheck implements the genericJSON Remote Procedure Call (RPC) specification providinga convenient interface for the RPC methods. The DApp canbe accessed through a regular web browser with an extensionof MetaMask. The MetaMask connects our DAPP to the mainblockchain provided by Ethereum. The MetaMask extensionutilizes the web3.js API to an injected web3 object, whichcan be used to access blockchain using asynchronous methodscalls.The
SkillCheck token contract is an
ERC-20 token. TheERC-20 token is a standard for implementing SCs on theEthereum blockchain. The following two sections describe theSCs we use in
SkillCheck .
1) College:
This segment of contracts contains two con-tracts, enrolment and wallet , that are all sub-standards of theERC-20 token. In the enrolment contract, various function-alities like sign-up and creation of wallets are implemented.The wallet contract is generated by the enrolment contract.Together, they are analogous to the admission process ofan institute, where each student/teacher/evaluator gets an IDcard having various functionalities. The whole platform uses
SkillCheck tokens, which is the currency of the paymentservice, and allows paying the transaction costs (gas) in thesetokens instead of the Ether. The wallet contract has all thefunctionalities of a usual Ethereum wallet. The wallet has theunique function of setting an admin role who can edit thescores (skill and evaluation) and transfer tokens. This roleis assigned to the instructor when (s)he is conducting theexam and is withdrawn after the exam is over. The processof assignment and withdrawal is actuated by the enrolmentcontract.
2) Test:
Test segment contains two SCs, exam factory and exam , that are called when the examiner wants toadd a new exam to the blockchain. Once the contract isadded to the blockchain, functions such as enrolintest,answermytest are called by the students to execute sim-ilar operations. The ( G ∗ , r ∗ , t ∗ ) algorithm is implementedand sends the computed marks to the exam contract, whichfurther distribute scores and rewards to the corresponding users’ wallet. The ERC-20 token style, which is followed in SkillCheck , allows the contracts to call the functions of eachother. However, these contract also have some private functionswhich allow certain users to take specific actions, e.g., onlythe examiner can take the final decisions on the grades.V. S
UMMARY
In this paper, we present a blockchain-based educationplatform
SkillCheck , which satisfies scalability, uniformityof scores over multiple evaluators, and ensures honest effortfrom the evaluators. We presented the detailed design of theplatform which is currently active internally in a universitynetwork. As a future work, we would like to investigate if suchguarantees can be ensured in general peer-review setups.A
PPENDIX
Proof of Theorem 1
According to the model (§III-B2), the instructor may notknow the true y j values of each paper j . This is becausewe assume that the student knows her y j perfectly and if r ∗ j (cid:62) y j , she does not raise a regrading request. Mechanism (cid:104) G ∗ , r ∗ , t ∗ (cid:105) will assume r ∗ j to be the true score and designthe evaluation scores accordingly when there is no regradingrequest. The student asks for regrading only if r ∗ j < y j .To prove Theorem 1, we show that the expected value of t ji is independent of b i and decreasing in σ i := τ i . The finalgrade after regrading is max { r ∗ j ( · ) , y j } = max { r ∗ j ( · ) − y j , } + y j . (3)We use the shorthand x = | P i | , ∀ i ∈ M since this number isa constant in our mechanism.According to the PG model of Piech et al. [12], ˜ y ( i ) j = y j + b i + n ij , where n ij ∼ N (0 , /τ i ) is a noise term. Hence,it is easy to show that ˆ b i = b i + (cid:80) k ∈ Pi n ik x and τ i = ˆ σ i = (cid:80) k ∈ Pi ( n ik − x (cid:80) n ik ) x , where n ik ∼ N (0 , /τ i ) .Substituting these values we get the expression for r ∗ j ( · ) − y j = √ γ ( µ − y j ) + (cid:80) l ∈ G ( j ) √ ˆ τ l ( n lj − (cid:80) k ∈ Pl n lk x ) √ γ + (cid:80) l ∈ G ( j ) √ ˆ τ l . With the assumption that y j is replaced with r ∗ j if notregraded, we can find the condition on y j when the R ( · , · ) function is non-zero to be y j (cid:62) √ γµ + Z + √ ˆ τ i ( n ij − (cid:80) k ∈ Pi nikx ) √ γ ,where Z = (cid:80) l ∈ G ( j ) \{ i } √ ˆ τ l ( n lj − (cid:80) k ∈ Pl n lk x ) . Note thatthe RHS of the inequality is independent of σ i . To find theexpected value of t ji , we need to integrate it w.r.t. y j , the limitsof which is, therefore, independent of σ i . By definition, the W ( − i ) ∗ j component of t ji is independent of bias and reliabilityof grader i . Hence, we only consider the first component whichis dependent on the bias and reliability of grader i . We willconsider the integral only w.r.t. y j to compute t ji and we justshowed that the limits of this integral is independent of σ i . As It is natural that a student getting a higher score than her expectation willnot contest the grading. ( r ∗ j , y j ) is decreasing in | r ∗ j − y j | , if we show that | r ∗ j − y j | is independent of b i and increasing in σ i , then we are done.It can be shown that r ∗ j − y j == Z − i (cid:113) (cid:80) k ∈ Pi ( m ik − x (cid:80) l ∈ Pi m il ) x · σ i + σ i · ( m j − x (cid:80) l ∈ P i m il ) X − i (cid:113) (cid:80) k ∈ Pi ( m ik − x (cid:80) l ∈ Pi m il ) x · σ i + 1 In the last equality, we substituted Z − i = [ √ γ ( µ − y j ) + (cid:80) l ∈ G ( j ) \{ i } √ ˆ τ l (˜ y ( l ) j − ˆ b l − y j )] and X − i = ( √ γ + (cid:80) l ∈ G ( j ) \{ i } √ ˆ τ l ) . Also, we substituted n ik = m ik · σ i . Since n ik ∼ N (0 , σ i ) , we get m ik ∼ N (0 , . We see that theabsolute value of the above expression is independent of b i and increasing in σ i . R EFERENCES [1] Ali Alammary, Samah Alhazmi, Marwah Almasri, andSaira Gillani. Blockchain-based applications in educa-tion: A systematic review.
Applied Sciences , 9(12):2400,2019.[2] Anujit Chakraborty, Jatin Jindal, and Swaprava Nath.Incentivizing effort and precision in peer grading. arXivpreprint arXiv:1807.11657 , 2020.[3] Bin Duan, Ying Zhong, and Dayu Liu. Educationapplication of blockchain technology: Learning outcomeand meta-diploma. In ,pages 814–817. IEEE, 2017.[4] Juan Carlos Farah, Andrii Vozniuk, Mar´ıa Jes´usRodr´ıguez-Triana, and Denis Gillet. A blueprint fora blockchain-based architecture to power a distributednetwork of tamper-evident learning trace repositories. In , pages 218–222. IEEE,2018.[5] Eric Funk, Jeff Riddell, Felix Ankel, and Daniel Cabrera.Blockchain technology: a data framework to improve va-lidity, trust, and accountability of information exchangein health professions education.
Academic Medicine , 93(12):1791–1794, 2018.[6] Alexander Grech and Anthony F Camilleri. Blockchainin education, 2017.[7] Jerinas Gresch, Bruno Rodrigues, Eder Scheid, Salil SKanhere, and Burkhard Stiller. The proposal of ablockchain-based architecture for transparent certificatehandling. In
International Conference on Business In-formation Systems , pages 185–196. Springer, 2018.[8] Jay Gupta and Swaprava Nath. Skillcheck: An incentive-based certification system using blockchains. Technicalreport, IIT Kanpur, 2020.[9] David Lizcano, Juan A Lara, Bebo White, and ShadiAljawarneh. Blockchain-based approach to create amodel of trust in open and ubiquitous higher education.
Journal of Computing in Higher Education , pages 1–26,2019. [10] Alexander Mikroyannidis, John Domingue, MichelleBachler, and Kevin Quick. Smart blockchain badgesfor data science education. In , pages 1–5. IEEE, 2018.[11] Jan Nespor. Cyber schooling and the accumulation ofschool time.
Pedagogy, Culture & Society , 27(3):325–341, 2019.[12] Chris Piech, Jonathan Huang, Zhenghao Chen, ChuongDo, Andrew Ng, and Daphne Koller. Tuned mod-els of peer assessment in MOOCs. arXiv preprintarXiv:1307.2579 , 2013.[13] Mike Sharples and John Domingue. The blockchainand kudos: A distributed system for educational record,reputation and reward. In
European Conference on Tech-nology Enhanced Learning , pages 490–496. Springer,2016.[14] Abhishek Srivastava, Pronaya Bhattacharya, ArunendraSingh, Atul Mathur, Om Prakash, and Rajeshkumar Prad-han. A distributed credit transfer educational frameworkbased on blockchain. In , pages 54–59. IEEE, 2018.[15] Peter Williams. Does competency-based education withblockchain signal a new mission for universities?
Journalof higher education policy and management , 41(1):104–117, 2019.[16] Yong Yuan and Fei-Yue Wang. Towards blockchain-based intelligent transportation systems. In , pages 2663–2668. IEEE, 2016.[17] Wenshuang Zhao, Kun Liu, and Kun Ma. Design ofstudent capability evaluation system merging blockchaintechnology. In