E-cheating Prevention Measures: Detection of Cheating at Online Examinations Using Deep Learning Approach -- A Case Study
JJOURNAL OF L A TEX CLASS FILES, VOL. XX, NO. XX, JAN 2021 1
E-cheating Prevention Measures: Detection ofCheating at Online Examinations Using DeepLearning Approach - A Case Study
Leslie Ching Ow Tiong and HeeJeong Jasmine Lee
Abstract —This study addresses the current issues in on-line assessments, which are particularly relevant during theCovid-19 pandemic. Our focus is on academic dishonestyassociated with online assessments. We investigated theprevalence of potential e-cheating using a case study andpropose preventive measures that could be implemented. Wehave utilised an e-cheating intelligence agent as a mechanismfor detecting the practices of online cheating, which iscomposed of two major modules: the internet protocol (IP)detector and the behaviour detector. The intelligence agentmonitors the behaviour of the students and has the abilityto prevent and detect any malicious practices. It can beused to assign randomised multiple-choice questions in acourse examination and be integrated with online learningprograms to monitor the behaviour of the students. Theproposed method was tested on various data sets confirmingits effectiveness. The results revealed accuracies of 68% forthe deep neural network (DNN); 92% for the long-short termmemory (LSTM); 95% for the DenseLSTM; and, 86% for therecurrent neural network (RNN).
Index Terms —Online examination, e-cheating detection,student assessment, intelligence agent, deep learning
I. I
NTRODUCTION O NLINE courses have become a feasible option ineducation. This platform is increasingly recognisedin colleges and higher education institutions such as uni-versities, and even implemented in elementary schools– for example, during the Covid-19 pandemic. However,the detached nature of online education raises concernsabout the potential risks of academic dishonesty, partic-ularly when students sit for exams at remote locations,in the absence of the disciplinary procedures that aretypically employed at examination centres [1], [2], [3].There is exponential growth in online education, in termsof both student enrolment and the corporate market itentails. However, existing literature indicate a prevalenceof online cheating, which involves academic dishon-esty by both the faculty and the students [4], [5], [6].Although online education provides valuable learningopportunities for people who do not have access totraditional quality education due to time or physical
L.C.O. Tiong is with Computational Science Research Center in Ko-rea Institute of Science and Technology (KIST), Seoul 02792, Republicof Korea (email: [email protected]).H.J. Lee is with Pierson College in PyeongTaek University,Pyeongtaek-si, Gyeonggi-Do 17869, Republic of Korea(email:[email protected]). constraints, its credibility may be compromised if issuesof academic dishonesty are not resolved.Although online courses have increasingly gained mo-mentum during the Covid-19 pandemic, it should not beestablished as an accepted mode of education withoutdue diligence in the event of a possible resurgence ofthe pandemic. For instance, the detached nature of onlinecourses has raised significant concerns as prevalence ofe-cheating has been reported [6]. This is because, whiletraditional sit-down examinations are invigilated, thesame cannot be said for the remotely conducted onlineexaminations. Consequently, the credibility of onlinecourses could become questionable.The present study aims to address the current limita-tions of cheating at online examinations by proposingartificial intelligence (AI) techniques via the internetprotocol (IP) network detector and deep learning-basedbehaviour detection agent. The research was conductedas a case study, the outcomes of which offer avenues offurther improving the intelligent tutoring system. Themain contributions of this study are as follows: • We analysed the limitations of the current onlineeducation system, with particular focus on cheatingat online examinations. • We proposed an e-cheating intelligence agent thatis based on the relationship model for detectingonline cheating using AI techniques. Specifically,we implement an IP detector and a behaviourdetector that utilises the long short-term memory(LSTM) network with a densely connected concept,namely DenseLSTM. As AI techniques have evolvedrapidly, and has been widely applied in recent years,we utilised state-of-the-art AI techniques for theonline exams, which may provide useful insightsthat contribute to the research area of an intelligenttutoring system. • We created a new dataset for this study, which ispresented in [7]. The records were collected acrossonline exams that were conducted in environmentsthat were highly unregulated (e.g., in the absenceof any implemented applications for detecting orpreventing e-cheating) during mock, mid-term, andfinal-term exam periods. The database includedtraining and testing schemes for performance anal-ysis and evaluation. a r X i v : . [ c s . H C ] J a n OURNAL OF L A TEX CLASS FILES, VOL. XX, NO. XX, JAN 2021 2
The paper is organised as follows: Section II analysesthe limitations of the existing online education systemfor preventing online cheating. The proposed frameworkis presented in Section III and the detailed databaseinformation is presented in Section IV. Section V dis-cusses the experimental results, and the conclusions aresummarised in the last section.II. B
ACKGROUND AND R ELATED W ORKS
Students engaging in cheating during examinations isa prevalent phenomenon worldwide, regardless of thecountry’s stage of development. To this end, early workdone by Roger [8], suggested installing the proctoringsecurity program into computers, which enables con-tinuous monitoring of each student’s computer screenon an instructor control view. Similarly, Cluskey et al. [9], proposed an eight-step model for reducing potentialcheating by students, which incorporated several cri-teria that were contributory to the exam, such as theduration of the exam, the time allocated for answeringeach question, etc. However, this model has limitationsgiven that it required the students to use a specialbrowser in order to access the examination applications,and the consequent necessity for the teacher to changethe questions every semester by using the randomisedapproach [9].
A. Network Security Methods
The rapid growth of wireless communication technol-ogy enables users to remotely access digital resourcesat any time. Many researchers have focused their workon strengthening the security of online exam protocolsto counter the security breaches that could occur in theeducation sector. Bella et al. [10] proposed a new protocolfor online examinations, which bypassed the necessityof involving a trusted third party to maintain disci-pline. The proposed protocol merged oblivious transferand visual cryptography that allowed both the studentsand invigilators to generate aliases. These would onlybe revealed during the exam, thereby maintaining theanonymity of the students.A literature survey by Ullah et al. [11] discusses thesecurity threats that have been encountered in the past,which are associated with online examinations. Theyindicated collusion to an be increasingly challengingthreat, which typically involves the collaboration of athird party who assists the student by impersonatinghim or her online. A further study conducted by thesame authors [12] revealed the potential mechanismsof security attacks in online cheating. They monitored31 online participants at an examination, where theyassessed the students’ behaviour by employing dynamicprofile questions in an online course. The results pointedout that students who cheated by impersonation sharedmost of the information using a mobile device, and con-sequently, their response time was significantly differentto those who did not cheat. A recent literature review [13] highlights the signif-icance of advanced technology in enabling the use oftechniques such as anomaly detection for addressingthe growing concerns of e-cheating. The authors re-viewed the current work under five dimensions: networkdata type, network traffic, intrusion detection, detectionmethods and open issues. They also emphasised theaccurate identification of an individual to be crucial indeveloping any intrusion detection system that is aimedat restricting their access within the network traffic.
B. Plagiarism Detection Methods and Tools
Plagiarism detection tools are popular in course eval-uations for identifying the unpermitted use of writtencontent by students. For instance, the code plagiarismtool can be used to calculate the similarity betweena pair of programs using a token sequence [14] anddependency graph features [15]. Nonetheless, these codeplagiarism detection techniques can be circumvented bymodifying the code syntax. In order to counter suchdeceit, Herrera et al. [16] implemented a new language-agnostic methodology, which prevents plagiarism inprogramming courses without the necessity for codecomparison or professor intervention.A study carried out by Pawelczak [17] examinedthe achievements and opinions of students over fiveyears regarding the automated evaluation system usedin plagiarism detection in programming courses. Thedata comprised of 228 records, where the analysis wasbased on tokenizing and averaging several features ofthe source code. The study revealed a limitation in themethod: given that plagiarism detection is dependent onthe thresholding value that is used to calculate the levelof similarity in the content, thresholding issues in thisapproach could give rise to false measures of prevention.
C. Biometrics Methods
It is vital to ensure the presence of an examineethroughout the entire examination. The invigilation ofonline examinations is difficult, and the absence of aphysical invigilator gives rise to a higher possibility ofsuspicious conduct as well as cheating attempts takingplace. Several strategies have been proposed to countersuch fraudulent activities during examinations includingthe monitoring of yaw angle variations, audio presenceand active window capture. For instance, Prathish et al. [18] and Narayanan et al. [19] proposed to implement thefeatures of point extraction and yaw angle detection thatcould assist the instructors in monitoring the studentsduring online examinations. Similarly, Wlodarczyk et al. [20] presented the head pose detection method, whichuses the precise localisation of face landmark points thathelp in identifying the user’s direction of gaze as wellas facial recognition.Hu et al. [21] proposed a novel method of monitor-ing the students’ behaviour during online assessments,
OURNAL OF L A TEX CLASS FILES, VOL. XX, NO. XX, JAN 2021 3 which involved identifying the relevant relationship be-tween the image of a student’s face and his or hercorresponding pose. Mahadi et al. [22] suggested thatthe combination of facial recognition and keystroke dy-namics could be the best classifiers for behavioural-basedbiometric authentication. In similar a study, Ghizlane et al. [23] primarily focused on the identity and accessmanagement of students and staff, where they coulduse a customised model of a smart card-based digitalidentity control system for accessing academic services,especially online evaluations. This service would help inensuring that the security of the institute could managethe authorisation of individuals in accessing varioustypes of networks.Ghizlane et al. [24] also examined the combined useof smart cards and face recognition to authorise andmonitor applicants while online exams are conductedin detecting any suspicious behaviour or cheating at-tempts. They recommended a system that stored a log ofphotographs taken of each applicant that sits the exam,which could be later checked by administrators. Anotherstudy conducted by Garg et al. [25] proposed usingthe convolutional neural network and the Haar cascadeclassifier to detect the faces of the exam candidates andto tag these with an associated name that is given atthe time of registration, which allows the system to keeptrack of the applicants’ movements within the timeframeof the exam.In current online examination settings, biometric au-thentication is considered to be one of the most pop-ular techniques in verifying the identification of thecandidates [26], [27]. In contrast to face-to-face exami-nations, online assessments do not involve proctors orinvigilators, and can be held in different and uncon-trolled remote environments. Consequently, establishingauthentication goals in online exams are vital in order toverify the identity of the students, as it plays a key role inonline security [28], [29]. In a study which focused on en-hancing the security of online examinations, Mathapati etal. [30] proposed utilising personal-images as graphicalpasswords. They suggested using digital pictures thatwere captured from live video as personalised physicaltokens.Ramu et al. [31] and Mungai et al. [32] reviewed theimportance of keystrokes dynamics in preserving secu-rity in online examinations. The proposed architectureused a three-stage authentications process, in which thestages were described as statistical, machine learningand logic comparison. Initially, when an applicant signsinto the system, his/her typing style is automaticallyrecorded, for which a template is generated. These tem-plates are subsequently used as a guide to continuouslymonitor the authenticity of the users, based on severalparameters: their dwell time (the time difference be-tween keypress and release); the flight time (the timepassed between the key release and keypress of twoconsecutive keystrokes); and, the typing speed, for betterprecision and robustness. Ananya and Singh [33] also introduced the keystroke dynamics approach, in whichthe system does not require any pre-registration and hasthe capability of keeping track of each student’s typingpattern during the exercise session itself.
D. Summary
Due to the Covid-19 pandemic, many academic insti-tutes, schools and universities have switched to onlineteaching, and consequently, online examinations havebecome a common trend, especially due to its flexibilityand usability within different environments [34], [35],[36]. Several studies offer diverse approaches to mitigatecheating at online examinations that largely focus onbehaviour analysis, technology innovation, etc. [37], [38],[39]. However, the detection of suspicious behaviourof candidates at online examinations still remains oneof the major challenges in fully utilising online edu-cation platforms. In this context, we offer a solutionfor detecting abnormal behaviour of students at onlineexaminations, and thereby for preventing e-cheating, byinvestigating the use of an AI intelligence agent as a real-time live proctor. The AI intelligence agent was designedutilising network protocol detection and deep learningapproaches. III. M
ETHODOLOGY
We designed an online examination as a case study,which consisted of multiple-choice questions, in whichan e-cheating intelligent agent was used to detect anypotential cheating. The e-cheating intelligence agent con-sists of two main agents: the network IP detection agent(described in Section III-A) and the behaviour detectionagent (described in Section III-B). Fig. 1 illustrates thearchitecture of the proposed system.
A. Network IP Detection Agent
Emerging security analysis has raised awareness ofthe challenges of online learning and has captured therapidly increasing attention of researchers of developingnew e-learning assessment methods. The present issuesarises from the inadequate understanding of the securitydataset that is stored through network protocols andthe analyses of data that use semantic association andinference methods [40].The proposed model is a two-stage process (see Fig. 1).In the first stage, we propose applying an IP detectionagent to filter any deceitful activity. For example, thesystem can monitor the exam candidates’ IP addresses.Most routers allocate dynamic IP addresses, which arenumerical labels that are specifically assigned to eachdevice that is connected to a computer network. Thiswould enable the system to issue an alert if a studentchanged their computer device or their initial location.In the proposed method, there would be several setsof exam questions (such as Set A, B, C, etc.). At thestart of an examination, after verification, a student is
OURNAL OF L A TEX CLASS FILES, VOL. XX, NO. XX, JAN 2021 4
Online
Exam
E-cheating Intelligence Agent
IP Detection Agent (Rule-based Expert System)
IP: XXX.XXX.XXX.XXX
Behaviour Detection Agent (Deep Learning System)
Real time IP storage
Verify the IP address of exam environments
If IP is detected as similar from the storage, reassign the exam questions.Begin monitoringIf abnormal behaviour is detected, reassign the exam questions & send warning messages.
Students
Fig. 1. The proposed framework for the e-cheating intelligent agent system. This framework describes the entire process for applying thee-cheating intelligent agent to identify abnormal behaviour and to prevent e-cheating at online examinations. randomly assigned a set of questions for assessment(e.g., Set A). If any abnormal behaviour is detected, thesystem changes the questions randomly to another Set(e.g., Set B). Similarly, if an incoming student entershis/her credential to access the online exam platformusing an IP address that was previously recorded in theIP list as suspicious, the system will generate a differentset of questions to the student. Algorithm 1 summarisesthe entire process of the IP detection agent.
Algorithm 1
IP detection agent
Input:
IP address from a new examinee E Output:
Decision D //Initialisation Let IP DB be the list of real-time IP with N size if E is not in IP DB then Add E into IP DB return D with random sets to E elsefor i → N doif E is found in IP DB [ i ] thenreturn D with specific sets to E endendend B. Behaviour Detection Agent
We devised a behaviour detection agent via a deeplearning approach to monitor and analyse the behaviour of all the students. As illustrated in Fig. 1, the agentwould alert the instructors and immediately reassignthe remaining questions with a new set of questionsonly in instances where abnormal behaviour is detectedin the students during the examinations. The followingsub-sections provide a more detailed explanation of thebehaviour detection agent.
1) Data pre-processing:
Before training the behaviourdetection agent, we first transformed each raw datarecord into a one-hot encoded feature, which definesthe behaviour of the student during the examination.For instance, each raw data record contains the resultsof the 20 multiple-choice questions, the total time (inminutes) taken for answering the examination, and thefinal score. In this study, we defined the one-hot encodedinput feature as: R ∈ × N , where N =
23. Thefirst 20 elements represent the given answers of the 20questions as: [
1, 1, 1, 1, 1, 0, 1, 0, · · · , 0 ] where the values 1and 0 reflect whether the answer is correct or incorrect,respectively. The last three elements define the speedof answering the questions, whether fast, normal orslow. The last three elements are defined as the speedof answering questions as fast, normal or slow. Fig. 2summarises data pre-processing, where the raw data isprocessed into one-hot encoded features.Next, we explored how we could label the data recordto identify the students’ behaviours. Initially, we la-belled the records as one of two major categories ofbehaviours: ‘normal’ or ‘abnormal’. In defining ‘abnor-mal’ behaviour, we assessed the speed at which thestudents have answered in instances where the questionshave been answered 90% correctly, according to one-hotencoded features. If the speed was represented as too OURNAL OF L A TEX CLASS FILES, VOL. XX, NO. XX, JAN 2021 5
Q1 Q2 Q3 … Q20 Time (Minutes)
Total
Scores IP Given Answer Mark Given Answer Mark Given Answer Mark Given Answer Mark Given Answer Mark
Raw Data Record Correct Wrong
Fast Normal Slow
One-hot encoded features
Speed of answering questions
Fig. 2. Generation of the one-hot encoded feature. The example demonstrates the process of generating one-hot encoded features from ourdataset. fast or too slow, they were labelled as ‘abnormal’; incomparison, the rest of the samples were considered as‘normal’.When defining the speed of answering questions twofactors were considered: the number of questions an-swered and the level of difficulty of each question. Forexample, we observed that if the level of difficulty ofa question was defined as ‘easy’, most of the studentscould answer it within 10 to 20 seconds; if it was definedas ‘moderate’ or ‘high’, they required 30 to 40 secondsor 1-2 minutes, respectively. However, the specifics ofsuch labelling criteria are dependent on the subjects orthe courses that are evaluated.
2) DenseLSTM:
We propose applying a deep learningnetwork, namely DenseLSTM as the behaviour detectionagent. The LSTM network was introduced by Hochre-iter and Schmidhuber [41], and allows modelling theproblem with sequential dependencies data, such astime-series data, natural language processing, behaviouranalysis, etc. In our work, we propose to use a denselyconnected approach with the LSTM network to extractbetter feature representation for abnormal behaviourprediction.Fig. 3 illustrates the architecture of the DenseLSTM,which consists of a convolutional ( conv ) layer, two LSTMblocks and a transition layer. The concept of the denselyconnected network was originally proposed by Huang etal. [42] for image classification. The network introducesdirect connections from any layer to all the subsequentlayers, which improves the information flow betweenthem by creating a different connectivity pattern. Oneexplanation for this occurrence is the enhanced accesseach layer has to all the preceding feature maps in itsblock due to the dense connectivity, and the “collectiveknowledge” it thereby provides the network.In the LSTM block, the LSTM cell layer decides whatinformation will be discarded from the cell state. Asthe layer could potentially inadvertently omit usefulinformation, we have implemented the concept of adensely connected network, which keeps the informationtogether instead of deciding “what to forget and whichnew information should be added.” The features canbe accessed from anywhere within the network, and
LSTM Blocks C on v2 × L S T M C e ll C on v2 × L S T M C e ll C on v2 × L S T M C e ll C on v2 × L S T M C e ll LSTM Blocks C on v × C on v1 × av gpoo l s o ft m ax Transition layerInput
LSTM Blocks f l a tt e n Fig. 3. Architecture of the DenseLSTM.TABLE IT
HE CONFIGURATIONS OF EACH LAYER FOR THE PROPOSED NETWORK . f REFERS TO THE SIZE OF FEATURE MAPS AND k REFERS TO THE SIZE OFFILTER . Network Layer Configuration conv f : 64@1 × k : 1 ×
2; stride 2
LSTM _ Block (cid:34) LSTM _ Cell × conv (cid:35) × transition × conv ; avgpool ; stride 2 LSTM _ Block (cid:34) LSTM _ Cell × conv (cid:35) × LSTM _ Cell × f latten × × Y × C unlike in traditional network architectures, there is norequirement to replicate them from layer to layer. Thenetwork used in this study is expected to collect richinformation while maintaining a low complexity of fea-tures, which can result in achieving a better classificationperformance.Let B denote the LSTM block with l in H layers,composed of LSTM cell, conv layer, rectified linear unit(ReLU) [43] and dropout layers: B = H l ([ x , x , x , · · · , x l − ]) , (1)where x to x l − represent feature outputs and [ · ] isdefined as a concatenation operator. We define l as 4 OURNAL OF L A TEX CLASS FILES, VOL. XX, NO. XX, JAN 2021 6
TABLE IIT
EN SAMPLE RECORDS OF MOCK EXAM PERIOD FROM OUR DATASET . ID Q1 Ans. Q1 Score Q2 Ans. Q2 Score · · ·
Q20 Ans. Q20 Score Grade Time IP · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · in the first block and l as 8 in the second block. Atransition layer is implemented in the first block thatperforms 1 × conv and avgpool operations, where 1 × conv is defined as the filter size of the conv layer that is1 ×
1. Table I tabulates the architecture of the proposednetwork.In the training stage, we implement a softmax cross-entropy L of logit vector and the respective encodedlabel: L ( Y ) = − E ∑ i C ∑ j L ij log ( softmax ( Y ) ij ) , (2)softmax ( Y ) ij = exp Y ij ∑ Cj exp Y ij , (3)where L , E and C denote class labels, the number oftraining samples in Y and the number of classes, respec-tively.In summary, we have devised a network using theconcept of a densely connected approach to extract betterfeature representation and to strengthen the feature acti-vation of the network for predicting potential e-cheating.IV. D ATASET
We have designed a new dataset, namely, the 7wiseupbehaviour dataset, which consists of 94 student recordsthat were acquired from different end-of-term exami-nations at Pyeongtaek University in South Korea. Allrecords were collected during the Spring semester, 2020.
A. Collection Setup
In order to create our database, we utilised an e-classlearning management system (LMS), which is adoptedby the university. The exam module is crucial for theLMS system, which can be used to set up quizzesand exam questions. The module allows the lecturersto fill out forms that outline vital information such asthe exam name, time allocations (opening time, closingtime and length of the exam), grades, whether shufflingof questions is allowed, etc. They can add questionsfrom the question bank, using various in-built optionsfor quiz/test settings including the type of examina-tion questions – whether multiple-choice, true/false andshort answers, and also allocate marks for each question. The system further enables the staff to add resourcesincluding images or links, and to provide general feed-back. Once the exam is completed and submitted, thesystem automatically assigns marks for the questionsbased on the answers that were pre-determined by thestaff. Finally, the system generates a CSV file compiling alist of all the submissions and the candidates’ records in-cluding their names, IDs, the answers for each question,the scores for each question, the grades, the total timetaken for completion (in minutes), and the IP addresses.Table II shows several records of our dataset.
B. Training and Benchmarking Protocols
For training and benchmarking protocols, 94 recordseach were obtained from mock-exams, and the mid- andfinal-term exams, respectively. Notably, the records thatwere used for the training and benchmarking proto-cols did not overlap. When designing the protocol todevelop or train our model, we divided the data setfrom mock-exams for training and cross-validations at aratio of 80:20. In addition, due to the imbalanced natureof the dataset, we have applied a data augmentationapproach that generated an additional 60 samples repre-senting cases of abnormal behaviour. In the benchmark-ing scheme, the task was to determine the behaviourof the examinees based on their manner of answeringquestions. V. E
XPERIMENTS
We conducted several experiments to evaluate therelative performance between our network and otherbenchmark networks. All the configurations used for thenetworks are described in Section V-A and the experi-mental results are presented in Section V-B.
A. Experimental Setup1) Configuration of DenseLSTM:
The proposed networkwas implemented using TensorFlow [44]. For the config-uration, we applied a learning rate of 1.0 × − andthe AdamOptimizer [45], where the weight decay andmomentum were set to 1.0 × − and 0.9, respectively.In the experiments, the batch size was set to 32 andthe training was carried out across 250 epochs. The OURNAL OF L A TEX CLASS FILES, VOL. XX, NO. XX, JAN 2021 7 training was conducted using our database according tothe protocols set out in Section IV-B; it was performedby the NVidia RTX 2080 Ti GPU.
2) Configuration of Benchmark Networks:
We selectedseveral deep networks to evaluate the performance ofthe behaviour study: the Deep Neural Network (DNN)[46], the LSTM [47] and the Recurrent Neural Network(RNN) [48]. These approaches have been successfullyapplied in behaviour studies previously [49] and [50].When conducting the experiments, we tried our bestto implement and fine-tune the LSTM and RNN fromscratch following the recommendations of [47] and [48],respectively. The training was also conducted using ourdatabase and following the protocols in Section IV-B. Allthe training processes were performed using the NVidiaRTX 2080 Ti GPU.
B. Experimental Results
In order to evaluate the performance of our networkin a real-life scenario and in a more objective manner,we used the mid-term and final-term examinations dataand compared our experimental results with the resultsobtained from other benchmark approaches. As shownin Table III, of the different approaches tested, the highestaccuracy, of 95.32%, was achieved by our system foroverall performance, followed by the second-best, theLSTM system, which reached an accuracy of 91.89%. Ourmodel outperformed the existing benchmark models by3.5% in accuracy demonstrating its superiority to theother benchmark approaches.In identifying the source of the improvements in oursystem, we observed that the accuracy of our networkwas higher than 92% in analysing the behaviour of stu-dents during online exams for both mid-term and final-term examinations, with values of 97.77% and 92.86%,respectively (as shown in Table III). The LSTM system,which was the second-best, demonstrated comparableaccuracy scores of 94.49% for the mid-term and 89.29%for the final-term examinations. In contrast, the perfor-mance of the DNN approach was low with an accuracyscore of 82.74% for the mid-term examination, and only52.68% accuracy for the final-term examination. In ad-dition, our investigations of the DNN system revealederror rates of 2.23% and 7.14% for mid-term and final-term examinations, respectively, which was associatedwith false prevention that occurred as a result of severalstudents being extremely slow to select their answers.With the intention of refining our system further, wefocused on characterising its sensitivity and specificityby applying the parameters, Receiver Operating Charac-teristics (ROC) and Area Under the ROC curve (AUC).This enables summarising the trade-off between the trueand false positive rates of a model by using differentprobability thresholds. As shown in Fig. 4, our network(DenseLSTM) achieved the highest AUC values of 0.9972and 0.9760, for the mid-term and final-term examina-tions, respectively. The LSTM and RNN systems also
TABLE IIIP
ERFORMANCE EVALUATION OF THE MID - TERM AND FINAL - TERMONLINE EXAMINATIONS AND OVERALL . T
HE HIGHEST ACCURACYFIGURES OBTAINED ARE HIGHLIGHTED IN BOLD FONT . Networks Mid-term (%) Final-term (%) Overall (%)
DNN 82.74 52.68 67.71LSTM 94.49 89.29 91.89RNN 87.20 85.02 86.11
DenseLSTM 97.77 92.86 95.32 demonstrated a similar performance for the mid-termexamination by achieving AUC values of 0.9870 and0.9810, respectively. However, the AUC values achievedby both these approaches for the final-term examinationswere lower than 0.94. These comparisons demonstratethat our network has outperformed most of the otherbenchmark networks for classifying normal and abnor-mal behaviours of students at online examinations withlow sensitivity and high specificity.
C. Discussion
Our experimental analysis and results demonstratethat the proposed approach can successfully address thechallenges pertaining to cheating at online examinations,and in preventing such abnormal behaviour of students.The proposed intelligence agent framework utilises theIP detector and the DenseLSTM network, which enablemonitoring the students through network protocols andbehaviour analysis to identify any potential plagiarism.As the examinations were evaluated over two terms,our results have shown a consistently high accuracy. Theresults confirm the performance of the proposed networkto be superior in detecting abnormal behaviour at onlineexaminations due to its ability to better maintain “col-lective knowledge” from the extracted features withinthe subsequent layers, instead of “forgetting” the infor-mation. This supports our assumption that DenseLSTMperforms better than other neural networks.VI. C
ONCLUSION
Online learning is a new and exciting opportunity forstudents and education institutions, which is gainingmomentum. In today’s environment, e-learning presentsunique opportunities, but also unique challenges. Theprimary area of concern in online assessments is aca-demic dishonesty in the form of cheating, which stu-dents attempt to achieve employing numerous avenues.Therefore, it is the responsibility of the education insti-tutions to implement more effective measures to detectacademically dishonest behaviour. This paper discussesthe concerns around online cheating and offers plau-sible mechanisms of monitoring and curtailing suchincidences using AI technology.We have demonstrated the effectiveness of the pro-posed e-cheating intelligent agent, which successfully in-corporates IP detector and behaviour detector protocols.
OURNAL OF L A TEX CLASS FILES, VOL. XX, NO. XX, JAN 2021 8 (a) Mid-term examination (b) Final-term examinationFig. 4. Performance comparisons of the ROC curve for mid-term and final-term examinations.
The agent has been tested on four deep learning algo-rithms: the DNN, DenseLSTM, LSTM and RNN, usingtwo exam datasets (mid-term and final-term exams). Thehighest overall accuracy of 95.32% was achieved by theDenseLSTM. The average accuracy rate is 90%, whichis sufficient to alert the lecturers to review the examresult of concern. We intend to continue developing web-based e-cheating monitoring systems in the future. Suchsystems would potentially provide a user-friendly inter-face for tasks such as uploading exam results, choosingalgorithms and other features. The system will be testedamong lecturers of various subjects, where they wouldbe able to set specific features that they wish to monitorand for detecting abnormal behaviour of students. Theexisting agent will be improved based on their feedback.R
EFERENCES[1] M. A. Sarrayrih and M. Ilyas, “Challenges of online exam, per-formances and problems for online university exam,”
InternationalJournal of Computer Science , vol. 10, no. 1, pp. 439–443, 2013.[2] K. Jalali and F. Noorbehbahani, “An automatic method for cheat-ing detection in online exams by processing the student’s webcamimages,” in
Proc. 3rd Conference on Electrical and Computer Engineer-ing Technology (E-Tech 2017) , Tehran, Iran, 2017, pp. 1–6.[3] Z. Raud and V. Vodovozov, “Advancements and restrictions ofe-assessment in view of remote learning in engineering,” in
Proc.2019 IEEE 60th International Scientific Conference on Power andElectrical Engineering of Riga Technical University (RTUCON) , Riga,Latvia, 2019, pp. 1–6.[4] N. C. Rowe, “Cheating in online student assessment: Beyondplagiarism,”
On-Line Journal of Distance Learning Administration ,pp. 1–8, 2004.[5] J. Moten Jr., A. Fitterer, E. Brazier, J. Leonard, and A. Brown, “Ex-amining online college cyber cheating methods and preventionmeasures,”
The Electronic Journal of e-Learning , vol. 11, no. 2, pp.139–146, 2013.[6] C. Y. Chuang, S. D. Craig, and J. Femiani, “Detecting probablecheating during online assessments based on time delay and headpose,”
Higher Education Research & Development , vol. 36, no. 6, pp.1123–1137, 2017.[7] 7wiseup behaviour dataset. [Online]. Available: http://7wiseup.com/research/e-cheating/data/ [8] C. F. Roger, “Faculty perceptions about e-cheating during onlinetesting,”
Journal of Computing Sciences in Colleges , vol. 22, no. 2,pp. 206–212, 2006.[9] G. R. Cluskey, C. R. Ehlen, and M. H. Raiborn, “Thwarting onlineexam cheating without proctor supervision,”
Journal of Academicand Business Ethics , vol. 4, no. 1, pp. 1–7, 2011.[10] G. Bella, R. Giustolisi, G. Lenzini, and P. Y. A. Ryan, “A secureexam protocol without trusted parties,” in
Proc. International Con-ference on ICT Systems Security and Privacy Protection , Hamburg,Germany, 2015, pp. 495–509.[11] A. Ullah, H. Xiao, and T. Barker, “A classification of threats toremote online examinations,” in
Proc. IEEE 7th Annual InformationTechnology, Electronics and Mobile Communication Conference (IEM-CON) , Vancouver, BC, Canada, 2016, pp. 1–7.[12] A. Ullah, H. Xiao, and T. Barker, “A dynamic profile questionsapproach to mitigate impersonation in online examinations,”
Journal of Grid Computing , vol. 17, no. 2, p. 209–223, 2018.[13] G. Fernandes Jr. et al. , “A comprehensive survey on networkanomaly detection,”
Telecommunication Systems , vol. 70, p. 447–489,2018.[14] L. Mariani and D. Micucci, “Audentes: Automatic detection ofteNtative plagiarism according to a rEference solution,”
ACMTransactions on Computing Education , vol. 12, no. 2, p. 1–26, 2012.[15] K. Chen, P. Liu, and Y. Zhang, “Achieving accuracy and scalabilitysimultaneously in detecting application clones on android mar-kets,” in
Proc. 36th International Conference on Software Engineering ,Hyderabad, India, 2014, p. 175–186.[16] G. Herrera, M. Nuñez-del-Prado, J. G. L. Lazo, and H. Alatrista,“Through an agnostic programming languages methodology forplagiarism detection in engineering coding courses,” in
Proc. IEEEWorld Conference on Engineering Education (EDUNINE) , Lima, Peru,2019, pp. 1–6.[17] D. Pawelczak, “Benefits and drawbacks of source code plagiarismdetection in engineering education,” in
Proc. IEEE Global Engi-neering Education Conference (EDUCON) , Tenerife, Spain, 2018, pp.1048–1056.[18] S. Prathish, S. Athi Narayanan, and K. Bijlani, “An intelligent sys-tem for online exam monitoring,” in
Proc. International Conferenceon Information Science (ICIS) , Kochi, India, 2016, pp. 138–143.[19] A. Narayanan, R. M. Kaimal, and K. Bijlani, “Yaw estimationusing cylindrical and ellipsoidal face models,”
IEEE Transactionson Intelligent Transportation Systems , vol. 15, no. 5, pp. 2308–2320,2014.[20] M. Wlodarczyk, D. Kacperski, P. Krotewicz, and K. Grabowski,“Evaluation of head pose estimation methods for a non-cooperative biometric system,” in
Proc. 23rd International Confer-ence Mixed Design of Integrated Circuits and Systems (MIXDES) ,Lodz, Poland, 2016, pp. 394–398.[21] S. Hu, X. Jia, and Y. Fu, “Research on abnormal behavior detection
OURNAL OF L A TEX CLASS FILES, VOL. XX, NO. XX, JAN 2021 9 of online examination based on image information,” in
Proc. 10thInternational Conference on Intelligent Human-Machine Systems andCybernetics (IHMSC) , Hangzhou, China, 2018, pp. 88–91.[22] N. A. Mahadi et al. , “A survey of machine learning techniques forbehavioral-based biometric user authentication,”
Recent Advancesin Cryptography and Network Security , 2018.[23] M. Ghizlane, F. H. Reda, and B. Hicham, “A smart card digitalidentity check model for university services access,” in
Proc. the2nd International Conference on Networking, Information Systems &Security , New York, NY, USA, 2019, pp. 1–4.[24] M. Ghizlane, B. Hicham, and F. H. Reda, “A new model ofautomatic and continuous online exam monitoring,” in
Proc. 2019International Conference on Systems of Collaboration Big Data, Internetof Things Security (SysCoBIoTS) , Casablanca, Morocco, 2019, pp. 1–5.[25] K. Garg, K. Verma, K. Patidar, N. Tejra, and K. Patidar, “Convolu-tional neural network based virtual exam controller,” in
Proc. 4thInternational Conference on Intelligent Computing and Control Systems(ICICCS) , Madurai, India, 2020, pp. 895–899.[26] A. Fayyoumi and A. Zarrad, “Novel solution based on face recog-nition to address identity theft and cheating in online examinationsystems,”
Advances in Internet of Things , vol. 4, pp. 5–12, 2014.[27] N. A. Karim and Z. Shukur, “Review of user authentication meth-ods in online examination,”
Asian Journal of Information Technology ,vol. 14, no. 5, pp. 166–175, 2015.[28] R. Bawarith, A. Basuhail, A. Fattouh, and S. Gamalel-Din, “E-exam cheating detection system,”
International Journal of AdvancedComputer Science and Applications , vol. 8, no. 4, pp. 1–6, 2017.[29] S. G. A. Hadian and Y. Bandung, “A design of continuoususer verification for online exam proctoring on m-learning,” in
Proc. 2019 International Conference on Electrical Engineering andInformatics (ICEEI) , Bandung, Indonesia, 2019, pp. 284–289.[30] M. Mathapati, T. S. Kumaran, A. K. Kumar, and S. V. Kumar,“Secure online examination by using graphical own image pass-word schemer,” in
Proc. 2017 IEEE International Conference onSmart Technologies and Management for Computing, Communication,Controls, Energy and Materials (ICSTM) , Chennai, India, 2017, pp.160–164.[31] T. Ramu and T. Arivoli, “A framework of secure biometric basedonline exam authentication: An alternative to traditional exam,”
International Journal of Scientific and Engineering Research , vol. 4,no. 11, pp. 52–60, 2013.[32] P. K. Mungai and R. Huang, “Using keystroke dynamics ina multi-level architecture to protect online examinations fromimpersonation,” in
Proc. 2017 IEEE 2nd International Conference onBig Data Analysis (ICBDA) , Chennai, India, 2017, pp. 160–164.[33] Ananya and S. Singh, “Keystroke dynamics for continuous au-thentication,” in
Proc. 2018 8th International Conference on CloudComputing, Data Science Engineering (Confluence) , Noida, India,2018, pp. 205–208.[34] M. A. Almaiah, . A. Al-Khasawneh, and A. Althunibat, “Exploringthe critical challenges and factors influencing the e-learning sys-tem usage during covid-19 pandemic,”
Education and InformationTechnologies , pp. 1–20, 2020.[35] M. H. Rajab, A. M. Gazal, and K. Alkattan, “Challenges to onlinemedical education during the covid-19 pandemic,”
Cureus , vol. 12,no. 7, pp. 1–11, 2020.[36] O. B. Adedoyin and E. Soykan, “Covid-19 pandemic and onlinelearning: the challenges and opportunities,”
Interactive LearningEnvironments , pp. 1–13, 2020.[37] N. A. Shukor, Z. Tasir, and H. Van der Meijden, “An examinationof online learning effectiveness using data mining,”
Procedia -Social and Behavioral Sciences , vol. 172, pp. 555–562, 2020.[38] N. Yan and O. T.-S. Au, “Online learning behavior analysisbased on machine learning,”
Asian Association of Open UniversitiesJournal , vol. 14, no. 2, pp. 97–106, 2019.[39] C. S. González-González, A. Infante-Moro, and J. C. Infante-Moro,“Implementation of e-proctoring in online teaching: A studyabout motivational factors,”
Sustainability , vol. 12, p. 3488, 2020.[40] Y. Yao, L. Zhang, J. Yi, Y. Peng, W. Hu, and L. Shi, “A frameworkfor big data security analysis and the semantic technology,”in
Proc. 2016 6th International Conference on IT Convergence andSecurity (ICITCS) , Prague, Czech Republic, 2016, pp. 1–4.[41] S. Hochreiter and J. Schmidhuber, “Long short-term memory,”
Neural Computation , vol. 9, no. 8, p. 1735–1780, 1997.[42] G. Huang, Z. Liu, G. Pleiss, L. Van Der Maaten, and K. Wein-berger, “Convolutional networks with dense connectivity,”
IEEE Transactions on Pattern Analysis and Machine Intelligence , p. 1–1,2019.[43] V. Nair and G. E. Hinton, “Rectified linear units improve restrictedboltzmann machines,” in
Proc. 27th International Conference onMachine Learning (ICML) , Haifa, Israel, 2010, pp. 1–8.[44] Tensorflow. [Online]. Available: https://tensorflow.org[45] D. P. Kingma and J. L. Ba, “Adam: A method for stochasticoptimization,” in
Proc. International Conference on Machine Learning(ICML) , Lille, France, 2015, pp. 1–15.[46] N. Kriegeskorte and T. Golan, “Neural network models and deeplearning,”
Current Biology , vol. 29, no. 7, pp. R231–R236, 2019.[47] K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, andJ. Schmidhuber, “LSTM: A search space odyssey,”
IEEE Transac-tions on Neural Networks and Learning Systems , vol. 28, no. 10, pp.2222–2232, 2019.[48] D. Rumelhart, G. Hinton, J. Koutník, and R. Williams, “Learningrepresentations by back-propagating errors,”
Nature , vol. 323, p.533–536, 1986.[49] M. Zerkouk and B. Chikhaoui, “Long short term memory basedmodel for abnormal behavior prediction in elderly persons,” in
Proc. International Conference on Smart Homes and Health Telematics(ICOST) , New York, NY, USA, 2019, pp. 36–45.[50] Z. Hao, M. Liu, Z. Wang, and W. Zhan, “Human behavior analysisbased on attention mechanism and LSTM neural network,” in