A Computer Science-Oriented Approach to Introduce Quantum Computing to a New Audience
IIntroducing Quantum Computing to an Audience Beyond Physicists:A Case Study over 22 Workshops in 10 Countries ¨Ozlem Salehi a , Zeki Seskir b , and ˙Ilknur Tepe c a ¨Ozye˘gin University Computer Engineering Department, ˙Istanbul; b METU PhysicsDepartment, Ankara, Turkey; c QTurkey
ARTICLE HISTORY
Compiled December 8, 2020
ABSTRACT
Quantum computing is a topic mainly rooted in physics, and it has been gain-ing rapid popularity in recent years. A need for extending the educational reachto groups outside of physics has also been becoming a necessity. The purpose ofthis study is to demonstrate that a shift in educational mindset which considersquantum computing as a generalized probability theory rather than a field ema-nating from physics and the utilization of quantum programming as an educationtool is effective for introducing quantum computing to a wider audience. 317 par-ticipants with different educational backgrounds from 10 countries took part in thisstudy. Data is collected from 22 workshops organized and administered by QWorldinitiative, covering an educational material formulated according to this mindset.The increase in basic knowledge of quantum computing concepts is assessed usinga single group pre/post-test design. The results show that there is a significant in-crease in the knowledge levels of the participants on quantum computing conceptsregardless of their gender, age, educational level and background. The study showsthat hands-on programming based approach focusing on quantum computing as ageneralized probability theory is an effective way to introduce quantum computingto an audience with no background in quantum physics.
KEYWORDS
Quantum programming; quantum information; quantum computing; workshop;pre/post test; education
Contact Email: [email protected] a r X i v : . [ phy s i c s . e d - ph ] D ec . Introduction The idea of a quantum computer was first popularized by Richard Feynman near 40years ago (Feynman 1982). The concept of a second quantum revolution expandedthis idea to an array of quantum technologies (Dowling and Milburn 2003) thathas been utilized for almost two decades now. However, it has been only recentlydemonstrated that quantum computers could outperform even the strongest classicalsupercomputers on particular (albeit useless for now) tasks (Arute et al. 2019), whichwas considered as a milestone and noted as the point of “quantum supremacy”(Preskill 2011).These developments in the hardware realm are also supported by national ini-tiatives in the order of billion dollars by countries like UK (Knight and Walmsley2019), China (Zhang et al. 2019), Russia (Fedorov et al. 2019), and the US (Monroe,Raymer, and Taylor 2019; Raymer and Monroe 2019). Also on a regional level, theQuantum Flagship program of the European Union is the largest international initia-tive of this scale (Riedel et al. 2019). Most of these initiatives have strong focus onnot only research and development but also in commercialization of these technologies.This focus on commercialization and maturation of quantum technologies allowedhundreds of firms to start exploring the commercial opportunities presented by thesedeveloping set of technologies. As of 2020, there are more than 100 firms focused onquantum computing . It has been noted in the literature that expanding the generallevel of understanding on quantum technologies, especially quantum computing, isgaining importance for both societal impact (Vermaas 2017), and commercializa-tion purposes in terms of developing the necessary quantum workforce (NASEM 2019).In 2016 IBM launched the world’s first open-access quantum computing hardwarescheme called ’IBM Quantum Experience’ . Initially, the system was a circuitcomposer with a drag and drop interface. Later in 2017, Qiskit was released by IBMas a programming language built upon Python for programmers to write quantumalgorithms and run.In this line of thinking, the QWorld initiative has been launched with the missionof establishing “an open access and public global ecosystem for quantum technologiesand quantum software by the year 2025” . To involve more people in the field ofquantum computing, QWorld has adopted the idea of organizing introduction toquantum programming workshops, and used quantum programming as the toolto introduce quantum computing. As of November 2020, 37 workshops have beenorganized in 17 countries and 749 people have completed the workshops.The material developed by QWorld to be used during the workshops is calledBronze and it is intended for everyone with a background in linear algebra andprogramming experience in Python. Bronze introduces quantum computing as ageneralization of probabilistic computing, which is an approach adopted by computerscientists (Yanofsky and Mannucci 2008; Arora and Barak 2009) and also popularized https://quantumcomputingreport.com/privatestartup/ http://qworld.lu.lv/ https://gitlab.com/qkitchen/basics-of-quantum-computing
2y Scott Aaronson in his book ’Quantum Computing since Democritus’ (Aaronson2013). Leaving the underlying physics behind, this approach allows teaching quantumcomputing phenomena through abstract and linear algebraic concepts with which theparticipants are already familiar. Furthermore, the task-based programming approachhelps participants to learn through experience instead of the classical approachwhich first presents all information about the subject without leaving any gap forself-learning. Altogether, this methodology accelerates the learning process throughbypassing the re-education of participants on the physical phenomena and puts offthe discussions on the fundamental properties of physical reality such as determinismvs. uncertainty, realism vs. contextuality, and so on.Computer science oriented approach which views quantum computing as a gener-alization of probabilistic computing has probably been used in many lectures, andinformal educational materials such as ours. The topic of how to introduce quantumcomputing to a wider audience (especially computer scientists) has also been aroundfor quite a while (Mermin 2003), and new methods which aims making quantumcomputing more available for the high-school and early undergraduate students arebeing developed (Economou, Rudolph, and Barnes 2020). Yet, no studies discussingthis methodology on an empirical level with a wide range of participants with differentbackgrounds were found in the literature.Teaching quantum computing through task-based quantum programming hasalso been growing in popularity recently and there are some reported case studies(LaRose 2019; Mykhailova and Svore 2020; Carrascal, del Barrio, and Botella 2020).However, these works are mainly concerned with singular experiences in a limitedcontext, usually one or two semesters of undergraduate/graduate level of universitylevel courses and they do not undertake any study to assess the effectiveness of theapproach on the knowledge acquisition of the students. The main purpose of ourstudy is to argue for the effectiveness of introducing quantum computing to a wideraudience through a hands-on task based quantum programming approach consideringquantum computation as a generalization of probabilistic computing, hence bypassingthe requirement of previous knowledge on quantum physics as an entry barrier to thefield. To the best of our knowledge, this is the first empirical study of this size in thefield to measure basic knowledge acquisition in quantum computing.In this context, pre and post-tests are conducted and data is collected from 22workshops and 317 participants. Our first research objective is to test whether thereis a significant increase in the basic knowledge levels of the participants. Our nextresearch objective is to analyze data to see whether various factors such as gender,age, department and educational background have an impact on the learning process.Detailed analysis on these are presented in Section 5.The rest of the paper is organized as follows. In Section 2, we review some back-ground information about teaching quantum computing and programming. In Section3 we describe the material used at workshops in detail. In Section 4, we explain ourmethods for assessment and in Section 5, we present our results. We discuss the mainfindsings in Section 6, the threats to validity in 6.1, and provide some suggestions foradoption of our approach in 6.2. We conclude by Section 7.3 . Background
A recent study in which 21 companies from the quantum industry are interviewedreveals that coding is the most valued skill according to these employers (Fox,Zwickl, and Lewandowski 2020). The study further suggests that an introductorylevel quantum course focusing either on the hardware or software track to increasequantum awareness would appeal to a variety of students from different sciencemajors. A similar point is made by Peterssen (2020), which emphasizes the needfor training of engineers and programmers as specialists in Quantum Programmingand Software to prepare for the quantum workforce. Another study on quantumengineering conducted with 26 experts in the field emphasizes the need for increasing’quantum awareness’ for the quantum workforce of the future (Gerke et al. 2020),while arguing that this is mostly necessary for non-physicists. Similarly, concepts likequantum literacy (Nita et al. 2020), are introduced as reaching to wider audiencesbecome important. Anticipating these rapid developments, in 2017, a diverse groupof researchers from academia and industry published a document titled “QuantumSoftware Manifesto” (Ambainis et al. 2017). This document emphasizes ”educatingmore quantum programmers,” among other crucial points. The literature alsodemonstrates that quantum computation is still more closely related to quantuminformation theory concepts than experimental fields on a conceptual level of mappingfor quantum technologies related academic topics, but the academic literature isoverwhelmingly rooted in physics (Seskir and Aydinoglu 2019).Traditional quantum computing curricula do not incorporate quantum program-ming as it is a fairly new area. Nevertheless, number of such courses and educativeevents is increasing as quantum industry is emerging. In 2019, Microsoft preparedan introductory quantum computing course which focused at implementation ofquantum algorithms by Microsoft’s quantum programming language Q
3. Workshop Material
In this section we will go through the details of the Bronze material, the tutorial whichhas been used during the workshops.
Approach
Quantum computing courses are usually offered in physics and computer science de-partments at the graduate level. Most of the courses begin with teaching the conceptsfrom quantum physics and focus on the theoretical aspects, which make quantum com-puting a relatively hard subject to grasp. There are several studies reporting studentand teacher experience in learning and teaching quantum physics (Singh 2001; Ire-son 2000; Krijtenburg-Lewerissa et al. 2017; Mermin 2003; M¨uller and Wiesner 2002).In (Akarsu 2010), the three main problems in teaching quantum physics related tomathematical difficulties as identified by the professors are listed as • Students not being able to connect formal mathematical training with mathe-matical tools for quantum physics • Students struggling with the new mathematical notation used in quantumphysics • Students having problems with the mathematical formulations of quantumphysicsApart from mathematical difficulties, another problem stems from abstract physicalconcepts as quantum physics is less intuitive when compared to other physics courses.Bronze is designed to teach quantum computing from a linear algebra and computerscience perspective, by presenting quantum computing as a generalization of proba-bilistic and classical computing and abstracting the physical concepts through linearalgebra. This is discussed elegantly in (Aaronson 2013) through the following quote: “Quantum mechanics is what you would inevitably come up with if you started fromprobability theory, and then said, let’s try to generalize it so that the numbers we used o call “probabilities” can be negative numbers. As such, the theory could have been in-vented by mathematicians in the nineteenth century without any input from experiment.It wasn’t, but it could have been.” This approach has several advantages over the traditional approach as discussednext. In traditional quantum computing courses, the concept of superposition isintroduced following its historical roots, either through photon polarization or spin ofan electron. For the photon polarization case, the concept of light as an EM-wave withpropagation at a certain direction, and polarization on an orthogonal direction needsto be understood. This requires a diversion into the theory of light and how wavespropagate, which are not obvious for participants without a background coveringat least one semester of modern physics. Similarly, introducing superposition usingspin of an electron is discussed in physics course through Stern-Gerlach experiments,which have no further immediate conceptual use in quantum computing. Instead ofrelying on these, we focus on superposition as a property of an unit vector in a Hilbertspace, which is a particular vector space where vectors have a constant L .2. Content
Bronze material is composed of Jupyter notebooks and it uses Qiskit framework ofIBM as the quantum programming language. Among the other alternatives such asCirq, PyQuil, Q • Basics of Classical Systems • Basics of a Quantum Program • Basics of Quantum Systems • Quantum Operators on a Real-Valued Qubit • Quantum Correlation • Grover’s Search AlgorithmEach notebook consists of some theoretical background about the subject, pro-gramming, and conceptual tasks, accompanied by a notebook in which the solutionsfor the tasks can be found. Some of the first tasks involve only Python programmingand the Qiskit library is introduced gradually as new quantum computing conceptscome forward. Most of the conceptual tasks involve mathematical derivations whichshould be completed on paper. Overall, the focus is on practical concepts andquantum programming instead of theoretical proofs. Aimed to serve as introductorymaterial for a 3-day long workshop, some traditional topics such as Deutsch andBernstein-Vazirani algorithms are omitted.Next, we will talk about each section in more detail.The first section of the material introduces the basics of classical systems. Thesection starts with classical bits and concentrates on probabilistic bits and states.Vector representation for probabilistic states and their evolution through operatorsby vector-matrix multiplication is given through a game of coin flips. A notation forprobabilistic states is also introduced which builds the foundations of braket notation.The tasks in this section do not involve quantum programming. An example Taskfrom the probabilistic bit notebook is given in Figure 1. Some other tasks include7andom probabilistic state generation and simulation of a coin-flipping game andthere are also some tasks to be completed on paper such as finding the tensor productof two states.Figure 1.: Simulating a biased coin task from Coin flipping game notebookBefore moving onto the basics of quantum systems, a notebook describing how torun quantum circuits in Qiskit is provided.Then comes the section about basics of quantum systems. The section starts withan interferometer setup which is given in Figure 2. It is presented as a quantumcoin-flipping experiment, the quantum analog of coin-flipping previously introducedin the basics of the classical systems section. The following notebooks contain tasksthat ask for the implementation of the experimental setups Qiskit. The section alsocontains notebooks on quantum states, quantum operators, and superposition andmeasurement. 8igure 2.: Experimental setup from quantum coin flipping notebookAs the scope of Bronze is restricted to real numbers, the quantum states are elegantlyrepresented in the 2D plane. Tasks in the section include drawing random quantumstates in the 2D plane using Python. There is a section on quantum operators with real-valued entries; reflections and rotations involving both programming and visualizationtasks. Examples are given in Figure 3. (a) Visualization of random quantum states onthe plane (b) Hadamard operator as a reflection oper-ator
Figure 3.: Tasks about visualization of quantum states and quantum operators withreal entriesThe next introduced subject is multiple qubits and entanglement. Superdensecoding and quantum teleportation protocols are continually discussed alongside withentanglement. Tasks in the section ask for the implementation of both protocols.There also some tasks in which mathematical derivations should be performed suchas verification of the superdense coding protocol.The last section of Bronze is about Grover’s search and almost all previous contentbuilds the foundations for this section. The first notebook aims to introduce amplitude9mplification idea through an “Inversion about the mean” game whose outputs canbe seen in Figure 4, where the idea of query and inversion phases are applied on alist of elements. One qubit representation of Grover’s Search and implementationof query and inversion phases with phase kickback are the next topics that are covered.Figure 4.: Inversion about the mean game
4. Methods and Survey Design
This study aimed to investigate the effectiveness of teaching the basics of quantumcomputing through quantum programming by assessing the Bronze material basedworkshops through increase in knowledge of key concepts of participants. One grouppre/post-test design has been used for the study. The study consists of pre/post-testresults of 22 two/three day workshops organized by QWorld using the Bronze materialin 10 countries between May of 2019 and March of 2020, and was finalized due to theglobal COVID-19 outbreak (detailed information on workshops can be found in Table2). Out of 430 participants who have completed the workshops, 317 participants filledboth pre and post-tests.The workshops were organized mainly by the local members of the QWorldinitiative (QCousins ). Although the same material was conducted for each workshop,instructors and mentors were different. The workshops were free of charge, organizedby volunteering members and they were open to all willing participants. The numberof participants was limited to at most 40 for an effective classroom environment,where mentors can assist participants. For the workshops with more than 40 ap-plicants, a selection process needed to be implemented. As a first rule, applicantsfrom disadvantaged minorities and women were prioritized which was followed by anassessment of motivation (which was asked in the application form). Participants withno answers or very brief and generic responses like “I am curious” for the motivationquestion were not accepted in such cases. No restrictions on department or educationlevel were imposed for participants, which allowed people from different fields toparticipate in the workshops.Participants received before-workshop instructions on how to setup Qiskit, andseveral notebooks for self-study in order to get acquainted with the style of theworkshop. These were not always followed by the participants, and the early part ofthe first day is usually spent by mentors helping participants setting up Qiskit. Somesolutions to this problem such as utilizing tools like binder or Google Colab were https://qworld.lu.lv/index.php/qcousins/ https://mybinder.org/ https://colab.research.google.com/ Questions Answer TypeWhat are the two fundamental quantum phenomena that differentiates quantumcomputing from classical computing? ChecklistWhat is the programming language that Qiskit runs on? Multiple ChoiceMatch the quantum logic gates with their respective matrix representations (leaveempty if you don’t know the subject). MatchingWhich of the following elements are not necessary for quantum teleportation? Selectall that applies. ChecklistWhich quantum resource is used for superdense coding? Multiple ChoiceWhat is the common property of probabilistic bits and qubits? Multiple ChoiceWhat is Grover’s algorithm used for? Multiple Choice
The questions were developed by several organizers together to test whether theparticipants were learning the essential points of the material, hence we kept thequestions on a basic knowledge level. More elaborated tasks are included in theJupyter notebooks, and participants who’ve gone through the training are expectedto be able to complete all the tasks by themselves. There are seven questions, one onthe base programming language that Qiskit runs on (which is Python), and six on theconcepts of quantum computing such as quantum logic gates, qubits, teleportation,superdense coding, and Grover’s algorithm. The list of questions can be found ontable 1 (the full questionnaire and the points for each question can be found in theAppendix section).The same questions were asked on both of the pre/post-test. The demographic datais collected with the pre-test. In the post-test, we also asked the attendance level of theparticipants and feedback on the workshop to determine whether satisfaction levelshave any correlations with the level of increase in basic knowledge acquisition.
5. Results
In this section, the main findings of the analyses performed on the pre/post-test dataare provided. These findings cover the overall results, mean, gain score, and the nor-malized gain score for the entire set. Additionally, comparison with respect to gender,age, education level, and educational background (department) are given. The infor-mation on the location and dates of the workshops, the number of participants whohave completed the workshops, and the number of participants that filled both test11an be found on Table 2.Table 2.: Detailed information on workshops where the tests are conducted.
Country/City Date Number ofParticipants Number of FilledPre/Post-TestsTurkey/Ankara 2019:May 3-5 34 34Poland/Krak´ow 2019:May 20-22 11 10Poland/Warsaw 2019:May 25-26 25 16Czech Republic/Brno 2019:May 29-31 14 10Czech Republic/Brno 2019:Jun 17-19 21 14Slovakia/Kosice 2019:Jun 26-28 10 8Hungary/Budapest 2019:Jul 3-5 26 20Hungary/Budapest 2019:Jul 8-10 20 14Montenegro/Podgorica 2019:Jul 16-18 13 9Turkey/Istanbul 2019:Jul 19-21 22 21Bosnia and Herzegovina/Sarajevo 2019:Jul 23-25 8 6Slovenia/Ljubljana 2019:Jul 30-1 28 21Turkey/Ankara 2019:Aug 2-4 23 18Latvia/Riga 2019:Sep 14-15 14 12Turkey/Ankara 2019:Sep 20-22 16 10Italy/Verona 2019:Sep 24-25 15 14Latvia/Riga 2019:Sep 28-29 9 6Turkey/Istanbul 2019:Oct 4-6 19 16Turkey/Konya 2019:Nov 8-10 30 14Turkey/Istanbul 2019:Dec 6-8 36 19Turkey/Ankara 2020:Feb 22-23 20 18Turkey/˙Istanbul 2020:Mar 7-8 16 7Total 430 317
Overall knowledge acquisition
For each participant, basic knowledge acquisition is measured by the gain score, whichis the difference between the pre/post-test scores. Overall, the mean test score ofall participants has increased from 32.32 to 71.09 with a gain score of 38.77. A plotshowing the pre/post-test scores of all participants is given in Figure 5.12igure 5.: Pre/post test scores of all participantsNormalized gain score is introduced (Hake 1998) as a measure of change to assessthe knowledge of the students at the beginning and at the end of a physics coursewhen the same test is used. According to Hake (1998), normalized gain score ( n gain )is calculated as, n gain = µ post − µ pre − µ pre where µ pre and µ post are the average pre/post-test scores respectively. Normalized gainscore for all participants is calculated as 0.57. A table summarizing statistics aboutoverall scores is given in Table 3.Table 3.: Descriptive statistics of test scores. Pre-test Post-test Gain n gain Min 0 13 -36Max 100 100 90Mean 32.32 71.08 38.77 0.57SD 25.03 20.83 24.21
Normality of the gain scores is tested with D’Agostino Pearson test as the samplesize is large ( > p -score of 0 . > .
05 showing that the gain scores are distributednormal. Visual inspection of the Q-Q plot given which can be found in Figure 6 alsosupports test results. 13igure 6.: Q-Q Plot of the gain scores of all participants.We used paired t -test for hypothesis testing and the test result reveals that there isa significant difference between the pre/post-test scores. The results are summarizedin Table 4.Table 4.: Results of normality test and paired-t test on all data. D’Agostino Pearson test Paired t -testStatistics p -value Statistics p -value4.41 0.11 -28.51 2.335e-89 We have conducted Shapiro-Wilk normality test for the gain scores acquired at eachworkshop and used paired t -test or Wilcoxon signed rank test for hypothesis testingdepending on the normality of the data. Hypothesis testing yielded that there was asignificant increase in the test scores of the participants in all workshops. n gain scoresof the participants vary from 0.33 to 0.83. The results are summarized in Table 5 andplotted in Figure 7 . The results are visualized using violin plot, which is a combination of boxplot and kernel density estimateand not all components correspond to actual data points (Waskom 2016).
Pre-test Post-test Shapiro-Wilk Test Hypothesis TestingWorkshop n Mean SD Mean SD Gain n gain Statistics p -value Statistics p -valueAnkara May 34 41.71 31.91 66.71 23.94 25.00 0.43 0.97 0.44 -6.61 1.61E-07Krakow 10 40.60 20.87 67.60 14.26 27.00 0.45 0.95 0.66 -3.08 1.31E-02Warsaw 16 31.31 13.01 66.06 20.46 34.75 0.51 0.94 0.40 -7.71 1.00E-06Czech But 10 45.70 20.42 84.40 13.55 38.70 0.71 0.85 0.05 -6.42 1.22E-04Czech Brno 14 29.43 20.52 87.71 16.10 58.29 0.83 0.91 0.18 -8.13 2.00E-06Slovakia 8 33.63 17.74 76.75 15.36 43.13 0.65 0.96 0.78 -5.04 1.50E-03Budapest1 20 48.40 27.98 81.85 15.83 33.45 0.65 0.96 0.57 -5.97 1.00E-05Budapest2 14 43.21 28.25 88.64 12.85 45.43 0.80 0.95 0.50 -6.06 4.10E-05Montenegro 9 26.22 18.12 65.67 15.38 39.45 0.53 0.78 0.01 0.00 7.16E-03˙Istanbul 21 26.81 24.12 66.52 21.93 39.71 0.54 0.97 0.69 -7.63 2.40E-07Sarajevo 6 15.50 13.47 58.50 23.96 43.00 0.51 0.85 0.15 -4.78 4.95E-03Ljubljana 21 32.14 25.23 77.86 17.17 45.71 0.67 0.95 0.39 -11.47 3.03E-10Ankara Aug 18 23.17 13.26 62.33 22.00 39.17 0.51 0.95 0.43 -7.34 1.00E-06Riga High 12 8.83 4.47 58.75 20.87 49.92 0.55 0.97 0.89 -8.35 4.00E-06Ankara Sep 10 22.60 18.53 67.30 23.51 44.70 0.58 0.95 0.71 -9.55 5.00E-06Italy 14 74.07 20.97 88.86 17.82 14.79 0.57 0.94 0.38 -2.94 1.16E-02Riga Tech 6 6.33 7.53 72.00 16.83 65.67 0.70 0.80 0.06 -13.49 4.00E-05Kadık¨oy 16 28.38 20.78 75.06 18.32 46.69 0.65 0.91 0.13 -6.64 8.00E-06Konya 14 20.21 14.42 46.86 19.70 26.64 0.33 0.96 0.68 -5.48 1.06E-04¨Ozye˘gin 19 26.37 23.74 66.68 17.66 40.32 0.55 0.88 0.02 9.00 5.33E-04TEDU 18 22.67 12.67 66.17 12.17 43.50 0.56 0.92 0.15 -13.29 2.09E-10QWomen 7 24.57 14.12 72.14 13.50 47.57 0.63 0.89 0.26 -7.04 4.09E-04 Figure 7.: Violin plot for the test scores of all participants grouped into workshops.
Post-hoc analysis for gender and age
Among the 317 participants who both filled pre/post-test. gender data is available for310 of them. Summary of the results based on gender data is available in Table 6.Table 6.: Difference with respect to gender.
Pre-test Post-test Shapiro-Wilk TestGender n Mean SD Mean SD Gain score n gain Statistics p -valueFemale 83 26.11 22.12 66.54 20.60 40.43 0.55 0.97 0.06Male 227 34.97 25.90 72.80 20.67 37.82 0.581 0.99 0.07 Gain scores are distributed normal both for male and female participants and two15ample t test resulted in a p -value of 0.403 (statistics = -0.837) proving that there is nosignificant difference between male and female participants in knowledge acquisition.Box-plot for the pre/post-test scores grouped into male and female participants isgiven in Figure 8.Figure 8.: Box plot for male and female participants.Age information is available for 308 participants. To assess whether there is anysignificant difference in the gain scores depending on the age, participants are groupedinto 6 different age groups and the results are summarized in Table 7. Sample sizesfor the age groups 45-54 and 55-60 are too small to make meaningful inferences. Forthe remaining age groups, data suggests that the most effective learning is realized forthe age group 13-17, taking into account both the gain scores and normalized gain.Table 7.: Difference with respect to age. Pre-test Post-testAge group n Mean SD Mean SD Gain Score n gain With age as a grouping variable, not all gain scores are distributed normally ac-cording to Shapiro-Wilk normality test. Consequently, we applied Kruskal Wallis testwhich resulted in a p -value of 0.024 (statistics=12.896), exhibiting a significant dif-ference in the gain scores between the age groups. Box plot of pre/post-test scoresgrouped into age groups is given in Figure 9.16igure 9.: Box plot for age groups Educational background
We grouped participants into four categories based on their education level by takinginto account the highest degree they are pursuing or have obtained: high school, bach-elor, masters, and doctorates. Number of participants belonging to each category andsummary of the test scores is given in Table 8Table 8.: Difference with respect to education
Pre-test Post-testEducation n Mean SD Mean SD Gain Score n gain High School 14 15.50 17.22 69.57 22.63 54.07 0.64Bachelor 144 27.97 20.50 68.23 19.15 40.26 0.56Masters 74 36.34 25.65 72.60 21.84 36.25 0.57Ph.D. 71 44.31 29.34 77.69 21.52 33.38 0.60
When the gain scores are compared, it is observed that the average gain scoredecreases as the education level increases. For each group, Shapiro-Wilk test provedthat the distributions of the gain scores are normal. One-way ANOVA test showsthat there is a significant difference between the groups with a p -value of 0.016(statistics=3.51). When we also take into consideration the pre-test scores, we observethat high school students are still the most effective learners while the n gain scores ofthe other three groups are very close to each other.Considering the participants who are not high school students and those whoprovided information about their current or graduated departments, we have cat-egorized them into five: computer science, physics, engineering, science, and socialscience. Engineering category consists of the students with backgrounds in electricaland electronical engineering and other engineering departments. Those who do notstudy physics, computer science and engineering but a science related department iscategorized as science. Results are summarized in Table 9.17able 9.: Difference with respect to department. Pre-test Post-testDepartment n Mean SD Mean SD Gain score n gain CS 89 30.11 20.24 73.21 18.54 43.10 0.62Physics 62 39.16 26.01 78.27 19.02 39.12 0.64Engineering 47 29.26 25.28 68.02 21.26 38.77 0.55Science 32 28.75 19.07 68.78 20.57 40.03 0.56Social Science 7 30.14 24.90 67.43 17.10 37.29 0.53
Even though the participants with the highest gain score are those with com-puter science background, those with physics background have the highest n gain score.Shapiro-Wilk test is conducted for the gain scores of each group which reveals thatthe gain scores are distributed normal. One way ANOVA test resulted in a p -value of0.811 (statistics=0.397) suggesting that there is no significant difference between thegroups. Satisfaction results
Post-test consisted of 10 statements related to participant satisfaction measuredon a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).Responses of the 283 participants for each statement is visualized in Figure 10.Figure 10.: Questions and results of the satisfaction test attached to the post-test.18ean satisfaction scores are ranging from 4.17 to 4.83 and the details can be foundin Table 10.Table 10.: Mean scores and standard deviation for satisfaction questions.
Statement Mean SD1 4.63 0.662 4.17 0.953 4.58 0.714 4.83 0.565 4.83 0.566 4.70 0.667 4.59 0.718 4.64 0.639 4.73 0.5910 4.70 0.65
Pearson Correlation Coefficient between the gain scores and satisfaction scores foreach participant shows that no correlation exists between the satisfaction level of theparticipants and their gain scores and details are omitted here.
6. Discussion
Depending on our findings presented in the previous section, we can discuss severalpoints regarding the effectiveness of the mindset adopted in Bronze material forintroducing quantum computing to a wider audience through task-based quantumprogramming approach.The material has been effective in terms of increasing the basic knowledge level ofparticipants. Considering each workshop separately, we see that n gain scores vary ina range between medium and high gain and there is a significant increase in the gainscores in all workshops. We can conclude that the success of the workshops heavilydepends on the material as it is the only unchanging factor in 22 workshops organizedin 10 countries with different educators and mentors.The gender gap in STEM is a global problem at all levels of education and inthe workforce, and there are various projects developed in Europe trying to closethe gap including ones in the area of physics and programming (Garc´ıa-Holgadoet al. 2020). Even though some workshops are aimed only at women and womenare prioritized when the number of applications exceeds the quota, the number ofwomen participants who completed the workshops is approximately one-third of men.There is a significant difference in the pre-test scores of female and male participantsreflecting the gap while no significant difference in the gain scores between thetwo groups was evident. Overall, the results suggest that hands-on programmingexperience in learning quantum computing has been beneficial for both male andfemale participants, and contributes towards closing the gap.There are several reported experiences in teaching quantum computing to highschool students (Hughes et al. 2020; Ozols and Walter 2018). Both the analy- Mann-Whitney U test resulted in a p -value 0.002 (statistics=7416) n gain (excluding the groups withless than 10 participants) and high school students have the highest mean gainscore and n gain compared to other groups. We can safely conclude that for highschool students with a basic knowledge of programming and linear algebra, aneducational material adopting the mindset discussed in this study is an effectiveway of being introduced to quantum computing. The results also indicate that thegroups with less prior knowledge on the topic are the ones that benefited the most,which is an expected finding since the material is designed as an introductory material.As the questions on the test are about basic concepts in quantum computing andthe material teaches new concepts through programming, participants with physicsbackground have the highest mean pre-test scores and those with CS backgroundhave the highest mean gain scores. No significant difference in the gain scores wasfound between the participants with different backgrounds and this was not surprisingbecause the material was aimed to be beneficial for participants with differentbackgrounds.Further research can be conducted with novel survey design with questions aimedat distinguishing different types of backgrounds’ (i.e. physics and CS) contribution tolearning. Such an analysis might provide insight to developing materials in accordancewith the background of participants. Since the Bronze material targets the generalaudience, such a method was not considered.There were 10 statements about the satisfaction of the participants which wereanswered in the Likert scale between 1 and 5 and the averages were above 4.5 foreach statement except statement 2. Statement 2 is about whether the participantshad enough time to complete the tasks on their own and had an average score of4.17. This is understandable as the material is quite dense for 3 days and sometimesparticipants struggle completing the tasks. However, considering that the majorityof the participants were satisfied with the workload, we do not plan dilution of thematerial for future workshops. Threats to Validity
Having discussed the findings of the study, let us now consider the threats to thevalidity of the research. Validity can be divided into four categories (Wohlin, H¨ost, andHenningsson 2003): internal validity, external validity, conclusion validity, constructvalidity. Next, we will be exploring each one in more detail.Internal validity is concerned with outside factors that can affect the dependentvariable. In single group pre/post design, some of the most common threats tointernal validity are listed as history, maturity, instrumentation, statistical regressionand attrition (Price and Murnan 2004). Workshops were organized either on two orthree consecutive days and continued the whole day, not leaving place for any otherintervention that might result in history and maturation threat. Instrumentationthreat neither poses a challenge, as the performance evaluation metric given inthe Appendix section relies on objective measures. We have also considered the20ormalized gain scores together with the gain scores to overcome ”You can only goup from here” phenomenon. Attrition, which is also known as mortality, takes placewhen some of the participants drop out of the experiment. In our setting, out of 430participants who completed the workshops, 421 participants answered the pre-test,317 participants answered both tests, and we have discarded the test results of theparticipants who only answered one of the tests. When we compare the mean scoreof the pre-tests with and without discarding the ones who did not answer post-test,the mean scores are 32.32 and 31.11 respectively, not yielding any implication aboutthe pre knowledge of the participants who did not answer post-test.External validity refers to the generalizability of the results when the sameexperiment is run with different people, at different times and settings. Even thoughenvironmental factors may pose a threat to external validity, we have analyzeddata which we have collected from 22 workshops organized in 10 countries bydifferent educators and mentors. The repetitive nature of our data is evidence of thegeneralizability of our experiment results. A possible threat might be volunteer bias,which stems from the fact that filling the pre/post-test surveys was not compulsory.Previous studies evaluating volunteer bias have reported that volunteers differ fromthe rest of the sample in various means (Rosnow and Rosenthal 1976), and one mightargue that the participants who did not choose to answer are the ones with lowersuccess rates.Construct validity encompasses the extent to which the experiment and theperformance measure captures the construct to be assessed. In this study, we aimed tomeasure knowledge acquisition, which according to Bloom’s Taxonomy of EducationalObjectives is the most basic level (Bloom et al. 1956). Design of the test questionscan be a threat to construct validity. Further attention can be paid to assess thereliability of the test questions in measuring knowledge by using various statisticaltests Wells and Wollack (2003) and to improve the test quality in general.Conclusion validity is concerned with the ability to draw correct conclusions basedon the outcomes of the experiment. For hypothesis testing to decide whether therewas a significant difference, we first conducted normality tests and then applied paired t -test or Wilcoxon singed rank test depending on the normality of the data. For theremaining post-hoc analysis, we again conducted normality test followed by two sampletest, Kruskal Wallis and one-way ANOVA depending on the structure of the data. Suggestions
The conducted analysis shows that the approach undertaken in Bronze is an effectiveway to introduce quantum computing to a larger audience. Therefore, we would liketo highlight some aspects of the computer science oriented approach in introducingquantum computing to participants with diverse backgrounds.When a task-based material is followed for teaching quantum computing, partici-pants should be given enough time to think and complete the tasks on their own, asthe learning heavily relies on self practice in this setting. Mentors and solution hintsare useful additions, but time is the essential element here. This was also reflected inthe survey results of the satisfaction test, as discussed above.21ne important note to make here is that, although we adopt an approachthat aims to bypass the prerequisite knowledge on quantum physics as an entrybarrier to the field, this aspect is the main appeal of the field for some of thenewcomers. This was clear to us by the questions asked during the workshops(mainly by physics and electrics and electronics engineering students or graduates)on the physical implementation of the abstract topics we were covering, and thesame requests resonate in the qualitative suggestions that participants providedafterwards. Requiring prerequisite knowledge on quantum physics is an entry barrierto the field for many, and introducing quantum computing through the historicaldevelopment path it followed (i.e. via quantum information theory) is a restrictiveapproach for a wider audience appeal. However, participants that are interested inthe physical aspects of this field should also be considered while preparing sucheducational materials, and providing suggestions for further studies that dive intothe hardware aspect of quantum computing is a meaningful way to split the dif-ference between physics oriented and CS oriented introduction to quantum computing.Finally, while bypassing the prerequisite knowledge in quantum physics, it is entirelylikely to introduce new entry barriers in the process of developing an educationalmaterial. For the workshop material we had, it was the prerequisite knowledge inbasics of Python programming. Although the required level of knowledge was mainlyintroductory, it still appeared as a challenge for participants outside of the CS realm.Considering that our (and possibly the readers of this paper) aim is to reach andintroduce the widest audience to quantum computing, this topic of replacing one entrybarrier for another should be taken notice of. In our case, this is surpassed by the helpof on-site mentor assistance, but alternative methods (like using built-in interfaces )can also be adopted.
7. Conclusion
The purpose of the current study was to share a methodology in introducing quantumcomputing to a wider audience by on-site events through task-based quantum pro-gramming and computer-science oriented approach and investigate its effectivenessbased on the conducted pre-test post-test surveys.As the field of quantum computing emerges rapidly, and the need for a quantumworkforce for industry grows urgent (Ambainis et al. 2017; NASEM 2019), expandingbeyond the roots in physics (Seskir and Aydinoglu 2019) is becoming a necessity foreducational purposes. Studies focusing on doing this through quantum programmingstarted appearing in the literature recently (LaRose 2019; Mykhailova and Svore 2020;Carrascal, del Barrio, and Botella 2020), with promising results. However, all are fromsingle study cases, mainly from undergraduate/graduate courses in universities. Wehave expanded this to a study covering 22 workshops in 10 countries with 317 partici-pants from diverse backgrounds. The research method was single group pre/post-testdesign where participants answered the same test before and after participating in Some examples are: https://quantum-computing.ibm.com/ - http://quantumcomputer.ac.cn/ -https://algassert.com/quirk - https://quantumcomputing.com/ - https://oreilly-qc.github.io/ , which was planned to be held for 200participants but due to going online it hosted around 4000 participants from over 100countries. This is followed by the Qubit by Qubit initiative , which is an onlinelearning initiative providing near 7500 participants from around the globe free accessto an ”Eight-Month Intensive Quantum Computing Course”, sponsored by IBM .Assessing the effectiveness of such online events, and using the insight obtained fromthese assessments to improve the educational process is also important. Quantumcomputing is a relatively new field, quantum programming is even newer. Therefore, itis safe to assume that we still have much to learn on how to best adjust the introductorylevel materials for different audiences, mediums, and methods of teaching. Exploringsuch aspects can yield valuable outcomes that can be utilized in the path forward tothe quantum era.
8. Acknowledgements
The analysis parts of this work has been completed under the QIntern program orga-nized by QWorld, therefore we would like to thank to the organizers of the program.Additionally, we would like to extend our gratitude to all the organizers of workshopsthat help administer the pre/post-tests. We thank Dr. Ula¸s ˙Ilic and Franziska Gerkefor their feedback and suggestions. Finally, we would like to thank Dilan K¨ose, Dr.Abuzer Yakaryılmaz, and Sercan Erer for their invaluable contributions to the study.
References
Aaronson, Scott. 2013.
Quantum Computing since Democritus . Cambridge Univer-sity Press. . cambridge . org/de/academic/subjects/physics/quantum-physics-quantum-information-and-quantum-computation/quantum-computing-democritus?format=PB .Akarsu, Bayram. 2010. “Einstein’s redundant triumph quantum physics: An extensive studyof teaching/learning quantum mechanics in college.” Latin-American Journal of PhysicsEducation https://qiskit.org/events/summer-school/ . qusoft . org/wp-content/uploads/2018/02/Quantum-Software-Manifesto . pdf .Arora, Sanjeev, and Boaz Barak. 2009. Computational complexity: a modern approach . Cam-bridge University Press.Arute, Frank, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C Bardin, Rami Barends,Rupak Biswas, et al. 2019. “Quantum supremacy using a programmable superconductingprocessor.”
Nature
574 (7779): 505–510.Asfaw, Abraham. 2020. “Learn Quantum Computation Using Qiskit.” http://community . qiskit . org/textbook .Bloom, Benjamin S, MD Englehart, EJ Furst, WH Hill, and DR Krathwohl. 1956. “Taxonomyof educational objectives: Cognitive domain.” .Carrascal, Gin´es, Alberto A. del Barrio, and Guillermo Botella. 2020. “First experiences ofteaching quantum computing.” The Journal of Supercomputing https://doi . org/10 . .D’Agostino, Ralph B, Albert Belanger, and Ralph B D’Agostino Jr. 1990. “A suggestion forusing powerful and informative tests of normality.” The American Statistician
44 (4): 316–321.Dowling, Jonathan P, and Gerard J Milburn. 2003. “Quantum technology: the second quantumrevolution.”
Phil. Trans. R. Soc. A. .Economou, Sophia E., Terry Rudolph, and Edwin Barnes. 2020. “Teaching quantum informa-tion science to high-school and early undergraduate students.” .Fedorov, A K, A V Akimov, J D Biamonte, A V Kavokin, F Ya Khalili, E O Kiktenko,N N Kolachevsky, et al. 2019. “Quantum technologies in Russia.”
Quantum Science andTechnology https://doi . org/10 . .Feynman, Richard. 1982. “Simulating physics with computers.” Int J Theor Phys .Fox, Michael F. J., Benjamin M. Zwickl, and H. J. Lewandowski. 2020. “Preparing for thequantum revolution: What is the role of higher education?”
Phys. Rev. Phys. Educ. Res.
16: 020131. https://link . aps . org/doi/10 . . . .Garc´ıa-Holgado, Alicia, Sonia Verdugo-Castro, Carina Gonz´alez, M Cruz S´anchez-G´omez, andFrancisco J Garc´ıa-Pe˜nalvo. 2020. “European Proposals to Work in the Gender Gap inSTEM: A Systematic Analysis.” IEEE Revista Iberoamericana de Tecnologias del Apren-dizaje
15 (3): 215–224.Gerke, Franziska, Rainer M¨uller, Philipp Bitzenbauer, Malte Ubben, and Kim-Alessandro We-ber. 2020. “Quantum Awareness im Ingenieurwesen: Welche Kompetenzen werden in derIndustrie von morgen gebraucht?”
PhyDid B - Didaktik der Physik - Beitr¨age zur DPG-Fr¨uhjahrstagung . phydid . de/index . php/phydid-b/article/view/1034 .Hake, Richard R. 1998. “Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses.” American journalof Physics
66 (1): 64–74.Hughes, Ciaran, Joshua Isaacson, Anastasia Perry, Ranbel Sun, and Jessica Turner. 2020.“Teaching Quantum Computing to High School Students.” arXiv:2004.07206 .Ireson, Gren. 2000. “The quantum understanding of pre-university physics students.”
PhysicsEducation
35 (1): 15.Knight, Peter, and Ian Walmsley. 2019. “UK national quantum technology programme.”
Quan-tum Science and Technology .Krijtenburg-Lewerissa, Kim, Hendrik Jan Pol, Alexander Brinkman, and WR Van Joolingen.2017. “Insights into teaching quantum mechanics in secondary and lower undergraduateeducation.”
Physical review physics education research
13 (1): 010109.LaRose, Ryan. 2019. “Teaching quantum computing through programming.” May. https://medium . com/@rlarose 26759/teaching-quantum-computing-through-programming-799283c9769a .Mermin, N. David. 2003. “From Cbits to Qbits: Teaching computer scientists quantum mechan-ics.” American Journal of Physics
71 (1): 23–30. https://doi . org/10 . . .Microsoft. 2019. “Developing a Quantum-Ready Global Workforce.” https:// loudblogs . microsoft . com/quantum/2019/12/18/quantum-ready-global-workforce-microsoft-universities/ .Monroe, Christopher, Michael G. Raymer, and Jacob Taylor. 2019. “The U.S. NationalQuantum Initiative: From Act to action.” Science
364 (6439): 440–442. https://science . sciencemag . org/content/364/6439/440 .M¨uller, Rainer, and Hartmut Wiesner. 2002. “Teaching quantum mechanics on an introductorylevel.” American Journal of physics
70 (3): 200–209.Mykhailova, Mariia, and Krysta M Svore. 2020. “Teaching Quantum Computing through aPractical Software-driven Approach: Experience Report.” In
Proceedings of the 51st ACMTechnical Symposium on Computer Science Education , 1019–1025.NASEM. 2019.
Quantum computing: progress and prospects . National Academies Press.Nita, Laurentiu, Laura Mazzoli Smith, Nicholas Chancellor, and Hellen Cramman. 2020. “Thechallenge and opportunities of quantum literacy for future education and transdisciplinaryproblem-solving.” arXiv preprint arXiv:2004.07957 .Ozols, Maris, and Michael Walter. 2018. “The Quantum Quest.” . quantum-quest . nl/ .Perry, Anastasia, Ranbel Sun, Ciaran Hughes, Joshua Isaacson, and Jessica Turner. 2019.“Quantum Computing as a High School Module.” arXiv preprint arXiv:1905.00282 .Peterssen, Guido. 2020. “Quantum Technology Impact: The Necessary Workforce for Devel-oping Quantum Software.” In QANSWER , 6–22.Preskill, John. 2011. “Quantum computing and the entanglement frontier.” arXiv preprintarXiv:1203.5813 .Price, James H, and Judy Murnan. 2004. “Research limitations and the necessity of reportingthem.”
American Journal of Health Education
35 (2): 66.Raymer, Michael G, and Christopher Monroe. 2019. “The US National Quantum Initiative.”
Quantum Science and Technology https://doi . org/10 . .Riedel, Max, Matyas Kovacs, Peter Zoller, J¨urgen Mlynek, and Tommaso Calarco. 2019. “Eu-rope’s Quantum Flagship initiative.” Quantum Science and Technology .Rosnow, Ralph L, and Robert Rosenthal. 1976. “The volunteer subject revisited.”
AustralianJournal of Psychology
28 (2): 97–108.Seskir, Zeki, and Arsev Aydinoglu. 2019. “The Landscape of Academic Literature in QuantumTechnologies.” arXiv preprint arXiv:1910.06969 .Singh, Chandralekha. 2001. “Student understanding of quantum mechanics.”
American Jour-nal of Physics
69 (8): 885–895.Vermaas, Pieter E. 2017. “The societal impact of the emerging quantum technologies: a re-newed urgency to make quantum theory understandable.”
Ethics and Information Technol-ogy
19 (4): 241–246. https://doi . org/10 . .Waskom, Michael. 2016. “Seaborn Violinplot.” https://seaborn . pydata . org/generated/seaborn . violinplot . html .Wells, Craig S, and James A Wollack. 2003. “An instructor’s guide to understanding testreliability.” Testing & Evaluation Services. University of Wisconsin .Wohlin, Claes, Martin H¨ost, and Kennet Henningsson. 2003. “Empirical research methodsin software engineering.” In
Empirical methods and studies in software engineering , 7–23.Springer.Yanofsky, Noson S, and Mirco A Mannucci. 2008.
Quantum computing for computer scientists .Cambridge University Press.Zhang, Qiang, Feihu Xu, Li Li, Nai-Le Liu, and Jian-Wei Pan. 2019. “Quantum InformationResearch in China.”
Quantum Science and Technology . . AppendicesAppendix A. Pre/Post-Test The pre-test (titled as the “Participation Form”) contains the following section at thetop.
Dear participants,The information below is being collected to enable researchers to estimate disparitiesin the effectiveness of the workshop you are currently attending. Participation is basedon voluntariness. The information you are providing will be only used by researchers ina possible academic publication and only be shared with workshop organizers.Thank you for your participation.
The section is followed by the items • Please enter your participation ID no. (required) • Gender • Age • Write your current department(s) (or the ones that you’ve graduated most re-cently) (required)The education level is obtained via a table of the form Table A1. For the analysis,we accepted the highest ongoing or graduated level as the “Education” parameter forthe participant. Table A1.: Education Level
Ongoing GraduatedHigh SchoolBachelorMasterPhD
Rest of the form consists of the test questions.Q1. (15 pt) What are the two fundamental quantum phenomena that differentiatesquantum computing from classical computing? Check them from the list below. • Wave-particle duality • Photoelectric Effect • Probabilistic Bits • Quantum Random Numbers • Superposition • Schr¨odinger’s Cat • Superdense Coding • Quantum Teleportation • Entanglement • I don’t know this subjectAnswer: Superposition and Entanglement. Only the answers of participants thathave selected only these two were accepted, selecting only one was deemed wrong.Q2. (10 pt) What is the programming language that Qiskit runs on? • C++ 26 C • Java • Julia • R • Python • DOS • BASIC • Fortran • I don’t know this subjectAnswer: Python.Q3. (15 pt) Match the quantum logic gates with their respective matrix represen-tations (leave empty if you don’t know the subject).a b c d eIdentity (I) GatePhase-flip (Z) GateRotation with 45 degree in planeHadamard (H) GateNot (X) GateThe gates were given in the form of a, b, c, d, e with following order:1 √ (cid:20) − (cid:21) , (cid:20) (cid:21) , (cid:20) − (cid:21) , (cid:20) (cid:21) , √ (cid:20) −
11 1 (cid:21)
Answer: d, c, e, a, b. Here, each correct matching is worth 3 points.Q4. (15 pt) Which of the following elements are not necessary for quantum telepor-tation? Select all that applies. • Classical Channel • Single Qubit Operations • Entangled Pairs • Toffoli Gate • I don’t know this subjectAnswer: Only Toffoli gate is ‘not’ necessary, hence the answers containing anythingother than Toffoli gate was accepted as wrong.Q5. (15 pt) Which quantum resource is used for superdense coding? • Quantum Money • Probabilistic Bits • Quantum Information • Entanglement • I don’t know this subjectAnswer: Entanglement. 276. (15 pt) What is the common property of probabilistic bits and qubits? • The probabilities sum up to 1 • The values can be only real numbers • Destructive interference happens • L1 norm is always 1 • L2 norm is always 1 • There is no common property • I don’t know this subjectAnswer: The probabilities sum up to 1.Q7. (15 pt) What is Grover’s algorithm used for? • Period finding • Factorization for prime numbers • Unordered search • Distinguishing between balanced and constant functions ••