How Computers Work: Computational Thinking for Everyone
MM. Moraz´an and P. Achten (Eds.): Trends in FunctionalProgramming in Education 2012 (TFPIE 2012).EPTCS 106, 2013, pp. 1–19, doi:10.4204/EPTCS.106.1
How Computers Work:Computational Thinking for Everyone
Rex Page ∗ University of OklahomaNorman, OK, USA Ruben GamboaUniversity of WyomingLaramie, WY, USA
What would you teach if you had only one course to help students grasp the essence of computationand perhaps inspire a few of them to make computing a subject of further study? Assume they havethe standard college prep background. This would include basic algebra, but not necessarily moreadvanced mathematics. They would have written a few term papers, but would not have writtencomputer programs. They could surf and twitter, but could not exclusive-or and nand. What aboutcomputers would interest them or help them place their experience in context? This paper providesone possible answer to this question by discussing a course that has completed its second iteration.Grounded in classical logic, elucidated in digital circuits and computer software, it expands intoareas such as CPU components and massive databases. The course has succeeded in garnering theenthusiastic attention of students with a broad range of interests, exercising their problem solvingskills, and introducing them to computational thinking.
What would you teach if you had only one course to help students grasp the essence of computationand perhaps inspire a few of them to make computing a subject of further study? Assume they have thestandard college prep background. This would include basic algebra, but not necessarily more advancedmathematics. They would have written a few term papers, but would not have written computer pro-grams. They could surf and twitter, but could not exclusive-or and nand. What about computers wouldinterest them or help them place their experience in context?This paper discusses one of the many possible answers to this question. It describes experiences inteaching an honors course for students from a variety of disciplines at the University of Oklahoma. Thestudents have varied interests and come from all college levels, first year to fourth year. They can choosefrom many courses to satisfy their honors requirements, from Beatles History to Moby Dick to What isScience? This course, called “How Computers Work: Logic in Action,” has succeeded in getting theenthusiastic attention of some of these students and in exercising their problem solving skills.The course includes some computer programming, but does not dwell on it. Students get enoughexperience to know what software is, but not enough to take on serious software development projects.The material helps students understand what makes automated computation possible by expanding oncomputational principles and overarching insights rather than the details needed in the practice of engi-neering. Most of the students will not continue with additional study in computer science. They will notbecome practicing engineers in hardware or software development. ∗ This material is based upon work supported by the National Science Foundation under Grant No. 1016532. Exceptional athletes in American colleges sometimes enroll with the intention of turning professional after one year.Coaching vernacular for these athletes is “one and done.” The title of this section applies the term to students who may takeonly one course in computation.
HowComputers Work: Computational Thinking for EveryoneComputers are demystified. Students grasp the fundamentals that make automated computation pos-sible. We have some objective support for these claims from student assessments of the course and fromscores on examinations. Exam scores averaged over 90% in both offerings of the course, and we thinkthe exams call for comparable problem solving skills and insight about concepts than other exams wehave given in computer science courses over the years.Student assessments (which are discussed in more detail in Section 3) are mostly, but not uniformly,positive about the understanding of the workings of computers that they have acquired through theirstudies in the course. One student was disappointed that the course failed to discuss “the actual physicalmechanisms of computers and how the different parts (e.g. RAM, motherboard, etc.) work together.”Others were similarly surprised by the content of the course, but they were happy rather than disappointedwith what they learned. Based on the overall tenor of student assessments, we think students did acquirea perspective on computers that make them less mysterious. Our conclusion is guided more by our pastexperience in teaching computer science than in objective evidence.The honors course is an outgrowth of a course in applied logic required in the computer scienceprogram at the University of Oklahoma. Other papers have discussed the earlier course and supportingtools [1, 2, 3, 4, 5, 6, 7].The honors course includes more material on the big picture of computational thinking and morewriting projects (as opposed to problem solving projects) than the original logic course, but there isenough overlap to make it an adequate substitute, at least for good students. Topics from the requiredcourse that are omitted in the honors version to make room for more coverage of the big picture includeKarnaugh maps, quicksort, and a few other algorithms. Coverage of deductive reasoning is reduced inthe honors course, but equation-based reasoning is covered at about the same level as in the requiredcourse. These changes make it possible to include material on large applications such as Google’s use ofMapReduce [8] for rapid searches and Facebook’s Cassandra approach to the massive database problem[9]. The material gives students who are not majoring in computer science a leg up, but does not overlapdirectly with mainstream material in most computer science programs. Such programs, at least in theUnited States, usually discuss logic in a required course in discrete mathematics, but the material on logicrarely takes up more than 20% of the discrete mathematics course and usually does not discuss directlyin any detail connections between logic and the design of computer hardware and software.The big ideas in the honors course are1. the correspondence between digital circuits and formulas in logic,2. how abstractions facilitate combining solutions to small problems to form solutions to big ones,3. how algebraic formulas can specify computations,4. how models expressed in software capture the behavior of processes and devices,5. how important, complex algorithms derive from simple, definitional properties,6. how different definitional properties can produce the same results at vastly different computationalexpense,7. how computational expense makes some useful devices feasible and renders others infeasible,8. and how all of these ideas bear on the ability of computers to deal with information on the massivescale needed to provide services like search engines, internet storefronts, and social networks.The idea of deriving new properties by rigorous reasoning from definitional ones is suffused through-out the course. Algebraic equations specify software and digital circuits as formulas in logic, and logicexPageand Ruben Gamboa 3is employed to derive new equations. In this sense, logic forms the basis of both the objects under studyand analysis of those objects. These ideas have been around for a long time. John McCarthy wrote aboutthem extensively over a half-century ago [10]. Based on our observations of the understanding of the big ideas by most of the computer sciencestudents we encounter, we think that the honors students that decide to go further have a better initialunderstanding of the big ideas than many of the students who decide early on to focus their studies incomputing. One student commented that “As someone who was already acquainted with a traditionalprogramming language, I thoroughly enjoyed learning the methods and ideas presented in this class.”Another said the course “helped me understand how computers communicate through their languagesand axioms.” We do not think many incoming computer science students would mention axioms asan element of their understanding of computers. Several made comments indicating that the courseunlocked new parts of their brains, again an unusual take on a computing course compared to that ofmost computer science students. “How Computers Work: Logic in Action” has been offered twice, so far, as part of a collection of “per-spectives courses” in the Honors College at the University of Oklahoma. As one of many requirementsfor earning a degree with honors, students must complete two perspectives courses. The first offering ofthis perspectives course was in spring, 2011. By popular demand, a repeat offering took place in spring,2012, and the course is scheduled again for 2013.Perspectives courses in the Honors College are limited to nineteen students. Nineteen studentsenrolled in the 2011 offering, but two dropped the course after a few weeks. The 2012 offering wasoversubscribed at twenty students. One dropped, leaving a full contingent of 19 to complete the course.Eight of the total of 36 students in the two offerings of the course had major fields of study outsidescience and engineering: history, letters, philosophy, linguistics, economics, drama, psychology, andbusiness. Sixteen were majoring in science, eleven were engineering students, and one, a first-yearstudent, had not yet decided on a major. One of the engineering students was in computer science, andthree were in computer engineering. Engineering students scored, on the average, three percentage pointshigher on examinations than students in science and five points higher than students outside science andengineering.Almost 80% of the students (28 of 36) were in their first two years of college. About 60% of thestudents had some prior experience in programming. In most cases this was a course in high-school orcollege, but five students had been programming for more than two years. None had any prior experience In a sense the course originates from in-person interactions with McCarthy. He visited the University of Oklahoma for twodays in 1996 at the invitation of the School of Geology and Geophysics to speak about the sustainability of human progress. TheSchool was attracted to McCarthy by some material on his website that was favorable to their interest in expanding explorationfor oil. The geophysicists did not feel that they could productively entertain McCarthy for two days, so they asked the directorof the School of Computer Science (which happened to be one of the authors) to escort McCarthy for meals and to arrange forhim to deliver a separate seminar for the School of Computer Science. It was a thrilling opportunity.At breakfast one morning, the subject of introducing an undergraduate course on reasoning about software arose. McCarthyremarked that logic would be the essence of the course. This remark provided the inspiration for organizing the course withlogic as the main theme and with reasoning about software providing examples of logic in action. It was truly an “aha” momentleading, albeit by a long and circuitous route, to the course that this paper describes. The reason for the limit of nineteen, rather than twenty or some other number, has been lost. It was decided over fifteenyears ago, and no member of the current Honors College faculty or staff has been with the College that long.
HowComputers Work: Computational Thinking for Everyonein functional programming. Students with prior programming experience scored, on the average, fourpercentage points higher on examinations than students without programming experience.All students in the course are honors students, which means they held a grade-point average of atleast 3.4 (out of a possible 4.0) at the time of joining the honors program and would need to maintainat least that average to have an honors designation (“cum laude”) on their diploma at graduation. Inaddition, enrolling in honors courses suggests a high level of self motivation. Honors students tend to bewell-engaged in their studies, and they participate energetically in class. They ask interesting questions,and vague or sloppy answers seldom go unremarked.To summarize, about 20% of students enrolled in the course were majoring in humanities, socialsciences, or business. About 30% were engineering majors (mechanical, electrical, chemical, computerengineering, computer science), and about 45% were science majors (physics, chemistry, biochemistry,microbiology, meteorology). Two of the science students were simultaneously working on a degree inmathematics. Almost 80% of the students were in their first two years of college. Engineering studentsscored higher, on the average, on examinations than students in arts and sciences, but not by much.Among arts and science students, science students did marginally better, but the margin was only twopercentage points.
Near the end of the course, the university asks students to complete a 20-question assessment of theirexperience in each of their courses. The full record of student responses to these questions for bothofferings of the course is accessible online [11]. We provide a summary in this section that we thinkgives an accurate picture of the range of student opinion. About 70% of the students completed thequestionnaire in both offerings of the course. The questionnaire has fifteen questions requiring a rankingon a five-point scale (5 for “far above average” down to 1 for “far below average”) and five questionscalling for free-form comments.Assessments of both offerings of the course were mostly positive. The ratings in 2012 were a bithigher than in 2011. For example, the median responses to the questions “How intellectually stimulatingwas this course?” and “How much did this course help you develop your critical thinking skills?” were4 in 2011, but 5 in 2012. In both years the median rating was 5 for “Overall this course was h r i ,” but themean went from 4.75 in 2011 to 4.9 in 2012.Responses to the question “What were the specific strong points of the course?” included observa-tions about interesting, novel concepts and readings, the quality of posted lecture notes, and improve-ments in critical thinking coming from challenging concepts and homework. One student respondingto a question about the weak points of the course said that the rigor of the course was tedious at times.Another said that the overall goal of the course was unclear. Other comments pointed out that the con-cepts were difficult for people not familiar with computers, that there was too much jargon, and thatexplanations in the textbook were sometimes incomplete or unsatisfactory. Some students complainedthat the course did not cover as many big concepts and ideas or have as many in-class discussions asother perspectives courses in the Honors College.Another question asked for an overall opinion of the course. Most responses to the question werepositive: “This course was beyond excellent.” “It was a lot of work, and very difficult, but I enjoyed itand am glad I took it.” The course “encourages difficult, critical thinking.” There were three negative The percentages do not add up to 100 because one student had not yet chosen a major. exPageand Ruben Gamboa 5responses to the question. One student asked for more writing assignments to make the course more likeother perspectives courses, and two students wanted more material on the “big picture.”The Honors College administration was pleased enough with student reactions in both years to askfor another offering in 2013.
Much of the material in the course centers around equations. For example, software is introduced asoperations that transform operands to results. This is the standard viewpoint of functional programming.Of course any software (or hardware), transforms input signals to output signals, which makes it possibleto take the functional viewpoint regardless of paradigm. However, this is only a high-level picture, andthe details will differ substantially in different paradigms.In this course, all software is described in the form of equations. That is, all programs in the courseare functional programs. This is partly to keep the amount of material within reasonable bounds for aone-semester course, but mostly to simplify reasoning about software and digital circuits and to make itpossible to use a mechanized logic to formalize some of the proofs.Operations are described informally, through templates that relate inputs to results. For example wedefine the list constructor, cons , with the following informal equation.( cons x [ x x ... x n ] ) = [ x x x ... x n ] { cons } The traditional definition of cons would avoid informality by introduction the constructor and itsdestructors together and avoid the need for names of elements of the list:( first ( cons x xs )) = x { fst-id formal } ( rest ( cons x xs )) = xs { rst-id formal } We think it works better to start with a less formal approach and work towards greater formality. Thatis why our initial presentation of cons has a list template with a name for each list element.Informally, we use square brackets to delimit templates for lists and n to denote an arbitrary naturalnumber. In this case (and most cases) subscripts distinguish the names of individual elements in the list.In the above example [ x x ... x n ] stands for a list with n elements, and [ x x x ... x n ] is a list with n + x and whose subsequent elements are x , x , and so on up to x n .We put labels for equations on the right in curly braces. Later, we refer to an equation by its label.We would refer to the foregoing equations as { cons } , { fst-id formal } , and { rst-id formal } .To be consistent, we use informal list templates in our initial definitions of first and rest , as withdid with cons , instead of the more formal presentation in { fst-id formal } and { rst-id formal } .( first [ x x ... x n + ] ) = x { fst } ( rest [ x x ... x n + ] ) = [ x ... x n + ] { rst } We ask students to specify tests that the operations would have to satisfy if they were functioningcorrectly. The equations { fst-id formal } and { rst-id formal } provide an example of tests of this kind.Following our usual habits, we present the tests informally:( first ( cons x [ x x ... x n ])) = x { fst-id } ( rest ( cons x [ x x ... x n ])) = [ x x ... x n ] { rst-id } Students have no difficulty convincing themselves that if cons , first , and rest fail to satisfy eitherof the equations, { fst-id } and { rst-id } , there must be something wrong with at least one of the operators. HowComputers Work: Computational Thinking for Everyone In tandem with informal templates, we introduce a formal notation, DoubleCheck [2], to specify testsexpressing our expectations about the values that formulas stand for. This makes it possible for students to practice expressing their expectations in a form that allows thecomputer system to perform tests automatically using random data. (defproperty fst-id(x :value (random-integer)xs :value (random-list-of (random-integer)))(equal (first (cons x xs))x))(defproperty rst-id(x :value (random-integer)xs :value (random-list-of (random-integer)))(equal (rest (cons x xs))xs))
Some students complain about oddities in the notation: “ equal ” for “=”, prefix notation instead ofinfix, fully parenthesized formulas instead of relying on rules of precedence, etc. But, they accept thesethings as necessary for communicating with the computing system. We think one reason for uncomplain-ing acceptance is that students are struggling with much more difficult concepts, and details of notationprovide a welcome retreat into the trivial. Complaints fade quickly. There are plenty of more challenging things for the students to think about.Besides, having multiple notations for the same mathematical object is a theme that comes up again whenstudents see the correspondence between formulas in Boolean algebra and diagrams of digital circuits,with decimal numerals and binary numerals as representations of natural numbers, and in other examples.DoubleCheck properties employ mutable state and higher order operations to generate random valuesfrom type specifications. This is impossible in ACL2 because ACL2 does not support those languagecharacteristics, but Dracula [4] runs property-based tests in the Racket [12] environment (the successor toDrScheme [13]), which does support them. We do not explain the higher-order nature of DoubleCheckproperties in the course. Based on student reactions, it seems unlikely that adding such a discussionwould bring more clarity to the material. DoubleCheck is similar to QuickCheck [14], except that DoubleCheck is embedded in the Dracula environment for ACL2and QuickCheck is embedded in Haskell. DoubleCheck does not have the sophisticated facility for narrowing counterexamplesthat QuickCheck provides, which means the DoubleCheck would be much less effective for industrial-strength usage, but wehave found it to be effective for the purposes of our course. Sometimes students do not accept arbitrary concepts so easily. For example, some students object strenuously to the truthtable for the implication operator. The table is derived from the basic equations, so their objection in this matter is morephilosophical than technical. They think the truth table simply fails to capture the usual meaning of implication. Our defenseavoids philosophy altogether. We point out that we are not using logic for discussing issues from everyday life. We are usingit to explain the inner workings of digital circuits and software. The implication operation is useful in this enterprise, whetheror not its properties conform to normal expectations. Of course, we know that it does conform to standard uses in naturallanguage, but getting into a debate about that point does not do much to advance the educational goals of the course. More recently we have been using a programming environment called
Proof Pad developed by Caleb Eggensperger [15].It lacks some of the facilities of Dracula, especially a modules system for packaging and controlling the visibility of identifiers,but is easier to install and runs faster.
Proof Pad does provide adequate facilities, including DoubleCheck, for a course withlogic as its central theme. exPageand Ruben Gamboa 7
Inductive definitions appear painlessly in the context of the testing of expectations. An early example isan operator append for concatenating lists.( append [ x x ... x m ] [ y y ... y n ] ) = [ x x ... x m y y ... y n ] Students recognize that if the operator append failed to pass either of the following tests, { app1 } and { app0 } , it could not be functioning properly.( append ( cons x xs ) ys ) = ( cons x ( append xs ys )) { app1 } ( append [ ] ys ) = ys { app0 } Initially, we avoid viewing these equations as an inductive definition. They are billed as simple teststhat a correctly functioning operator would pass. Later, we assert that any such collection of equationsactually defines an operator, provided the equations have the following characteristics:1.
Consistent : no two equations specify different results for the same input;2.
Comprehensive : all forms of input must match the operands on the left side of at least one equation;3.
Constructive : any inductive reference to an operator must comprise a reduced computation.Students learn that under these conditions, all properties of the operator derive from the definitionalequations. In other words, the equations define the operator. We refer to these characteristics as “thethree C’s.” They are recurring theme from early on and throughout the course.We describe an inductive reference to an operator, op , as an invocation of op on the right-hand sideof an equation, { eqn } , that also refers to op on the left-hand side. The operands in the inductive referencemust match the operands on the left-hand side of a non-inductive equation more closely than they matchthe operands on the left-hand side of { eqn } . That is, the reference on the right-hand side of the equationrepresents a reduced computation compared to the formula on the left-hand side.We do not attempt a rigorous definition of the term “reduced computation.” We do not go into detailabout the degree of matching between operands, either. The reduced computation issue, especially, canbe subtle, but is not subtle in any of the inductive references used as examples in the course. In manycases the operands in inductive references are lists that are shorter than the corresponding operands onthe left-hand side of the equation, and the corresponding operands in non-inductive equations are of afixed, shorter length, usually zero, sometimes one or two. In other cases, an argument that is expectedto be a natural number is smaller in the inductive reference than on the left-hand side, and non-inductiveequations specify the corresponding operand as a fixed, smaller value, usually zero.Since students do not encounter in the course subtle reductions in the size of the computation ex-pressed by inductive references, we think that including a definition of computation size or degree ofmatching in operands would be more obfuscatory than helpful. Similarly, we believe that a discussion offixpoints would take us far afield of what we are tying to communicate in the course. Accordingly, wehave not tried to include these ideas in the discussion.However, offering students the opportunity to explore these ideas in one or more of the writingprojects they are required to complete as part of their course work could bring substantial rewards tocapable students. Probably, one-on-one discussions of the ideas during office hours would be a necessaryform of guidance, and that could be rewarding to both student and instructor. We are intrigued by thenotion of going in this direction in future offerings of the course.Another subtle issue has to do with matching in data structures and substitution of new, equivalentformulas into parts of existing formulas. We point out to students that this is a non-trivial activity and thatearly explorations in formal logic got the definition of matching and substitution wrong more than once HowComputers Work: Computational Thinking for Everyonebefore arriving at a correct definition. In this case, too, we rely on students to figure it out from practicerather from a formal definition, which we believe students would find more confusing than helpful.We think there is some evidence for our position in this matter in our discussion of the correspon-dence between digital circuit diagrams and Boolean formulas. We do present a formal definition of theisomorphism, but students routinely ignore it. They manage to work out correspondences between for-mulas and circuit diagrams in practice without referring to the formal definition. When they try to applythe formal definition, they usually get lost in details and technicalities. The append operator is associative.( append xs ( append ys zs )) = ( append ( append xs ys ) zs )Students express this property formally in the notation of the DoubleCheck facility. They run the testand find that it succeeds. (defproperty app-assoc(xs :value (random-list-of (random-integer))ys :value (random-list-of (random-integer))zs :value (random-list-of (random-integer)))(equal (append xs (append ys zs))(append (append xs ys) zs))) Of course the associativity property, like all properties of the append operator, can be derived fromits definitional properties, { app1 } and { app0 } . We derive properties of operations mostly by substitutingnew, equivalent, formulas at strategic points to form new equations, a method entirely familiar from high-school algebra, except that the operators involved, instead of being addition, multiplication and the like,often deal with non-numeric data, such as lists. Since the definitional equations are usually inductive,most of our derivations cite an induction hypothesis at some point to justify moving from one formula toanother, equivalent one.A pencil-and-paper proof of the associativity property from the informal equations could be carriedout as an induction on the length of xs . The base case, when xs is the empty list, cites the { app0 } equationtwice. ( append [ ] ( append ys zs ))= ( append ys zs )) { app0 } = ( append ( append [ ] ys ) zs ) { app0 } In the inductive case, the length of xs is non-zero. That is, xs = [ x x ... x n + ] for some naturalnumber n . So, the inductive case can be argued as follows.( append [ x x ... x n + ] ( append ys zs ))= ( append ( cons x [ x ... x n + ] ) ( append ys zs )) { cons } = ( cons x ( append [ x ... x n + ] ( append ys zs ))) { app1 } = ( cons x ( append ( append [ x ... x n + ] ys ) zs )) { ind hyp } = ( append ( cons x ( append [ x ... x n + ] ys ) zs )) { app1 } = ( append ( append ( cons x [ x ... x n + ] ) ys ) zs ) { app1 } = ( append ( append [ x x ... x n + ] ys ) zs ) { cons } The DoubleCheck property app-assoc is a formal statement of associativity, and students can directDracula to submit the property to the ACL2 theorem prover [16] for formal, mechanized verification.exPageand Ruben Gamboa 9In this way, students gain experience with inductive definitions and with expressing expectations bothformally and informally. The also learn to do informal, paper and pencil proofs, and they see how amechanized logic can be used to produce formal proofs. This is an important point, philosophically,because students observe that it is easy to get paper-and-pencil proofs wrong. Formalization is necessaryfor assurance that claims about properties of software or hardware are correct.
Suppose a student wants to define an operator that extracts the first n elements from a list xs . Using theprinciple of the three C’s described in Section 4.3, the student looks for equations that express propertiesof the operator that are consistent, comprehensive, and constructive. Students who manage to conjure upthe following equations have succeeded in writing a program for the prefix operator.( prefix xs ) = [ ] { pfx0 } ( prefix n [ ] ) = [ ] { pfx −} ( prefix ( n + ) ( cons x xs ) ) = ( cons x ( prefix n xs )) { pfx1 } We do not pretend that inventing these equations is easy. It calls for creativity and insight, and thosethings come from practice. Students need to do lots of exercises to learn the material. Gradually, theybuild up their expertise, and along with it comes the ability to derive new properties of operators fromdefinitional ones. The entire mechanism is built on ordinary, algebraic equations and classical logic. To run their program for the prefix operator, they must formalize it in ACL2 [16]. Since thereare three equations, the definition will need to say which formula for ( prefix n xs ) applies in whatcircumstances. It can use the “ if ” operator to make the appropriate selection. In this example, eventhough there are three equations, there are only two distinct formulas for the value of ( prefix n xs )because the empty list is the result that prefix delivers in two of the equations. So, the “ if ” operatoronly needs to choose between two formulas, and the definition could be written as follows. (defun prefix (n xs)(if (and (consp xs) (not (zp n)))(cons (first xs) (prefix (- n 1) (rest xs))) ; {pfx1}nil)) ; {pfx0} At this point, students can think about other properties of the prefix operator and test their ex-pectations with DoubleCheck. One property they might think of is a relationship between prefix and append . ( prefix ( len xs ) ( append xs ys )) = xs { app-pfx } A formal, DoubleCheck definition of this property would mechanize the test. One could provide guidelines beyond the three C’s, such as mimicking in function definitions the patterns of inductionfound in data structure definitions, following, for example, the pedagogy of the textbook
How to Design Programs [17]. How-ever, we do not expect students to become accomplished programmers. We want them to experience a few creative insights andto understand how computers can interpret algebraic equations as programs. We hope that a few of them may be inspired tostudy software development more seriously. At that point they can acquire a facility with design patterns that will enable themto be good programmers. ACL2 is based on Common Lisp. To run ACL2 programs, the Dracula programming environment converts them to Schemeand employs Racket [12] to interpret them. However, Dracula uses the ACL2 mechanized logic to prove theorems. (defproperty app-pfx ; preliminary version(xs :value (random-list-of (random-integer))ys :value (random-list-of (random-integer)))(equal (prefix (len xs) (append xs ys))xs))
All of the random tests that DoubleCheck generates pass, so the next step is a paper-and-pencil proofthat the property holds for all lists. That proof, an induction on the length of xs , succeeds.Unfortunately, ACL2 fails to complete a formal proof of the theorem corresponding to the app-pfx property. The problem is that, while the property holds for all of the random tests that DoubleCheckgenerates, it does not hold under all circumstances. If xs is not a list, but is, instead, some other kindof object, the property fails. To be an ACL2 theorem, the property must constrain xs to the domain oflists. (defproperty app-pfx ; provable version(xs :value (random-list-of (random-integer))ys :value (random-list-of (random-integer)))(implies (true-listp xs)(equal (prefix (len xs) (append xs ys))xs))) Even then, ACL2 does not succeed with a proof until it imports the standard theorems of numericalgebra, which have been derived in a certified package distributed with the ACL2 system [18]. So, inthis example, students must deal with a few of the complications that can arise when moving from aninformal environment to a formal one.One of the reasons we chose ACL2 for this course rather than another mechanized logic, such asIsabelle [19], Coq [20], or Agda [21] is because the learning curve for ACL2 allows students to succeedquickly in problems of moderate complexity. Other theorem proving systems may be equally effective.We have not tried others in the classroom, so we are guessing based on our own assessment of thedifficulty of applying other systems in the types of examples that we use in our course.Our guess is based in part on the collections of powerful theorems that come with ACL2. Studentsimport theorem collections when ACL2 gets stuck. The two collections they learn to import are the onesthat imbue ACL2 with facilities for manipulating formulas in numeric algebra. One of the collectionsconcerns ordinary arithmetic, the other, modular arithmetic. These collections of theorems provide thesupport needed to complete homework projects.Examples and projects in the course progress from simple ones like prefix to complex ones likemerge-sort and AVL tree operations. All of them include testing and deriving, informally and formally,additional properties from definitional ones. Students are not asked to create programs of this complexityfrom scratch, but they are required in homework projects and on exams to extend properties to widerdomains and to develop new properties. They are given guidelines for developing such properties [22].
Equations provide a basic theme that permeates the course. Because the equations of Boolean algebraare so much like those of numeric algebra, which students are familiar with, the technical part of the This is, of course, a vestige of type, and we do not talk much about types. Our students have raised questions on many topicsin the course, but none have ever asked about types, so we think our decision to avoid the issue and use ad hoc explanationswhere necessary works in the ACL2 environment, at least at the level presented in this course. exPageand Ruben Gamboa 11 x ∨ False = x {∨ identity } x ∨ True = True {∨ null } x ∨ y = y ∨ x {∨ commutative } x ∨ ( y ∨ z ) = ( x ∨ y ) ∨ z {∨ associative } x ∨ ( y ∧ z ) = ( x ∨ y ) ∧ ( x ∨ z ) {∨ distributive } x → y = ( ¬ x ) ∨ y { implication }¬ ( x ∨ y ) = ( ¬ x ) ∧ ( ¬ y ) {∨ DeMorgan } x ∨ x = x {∨ idempotent } x → x = True { self-implication }¬ ( ¬ x ) = x { double negation } Figure 1: Basic Boolean equations (axioms) ( x ∨ y ) ∧ y = ( x ∨ y ) ∧ ( y ∨ False ) {∨ identity } = ( y ∨ x ) ∧ ( y ∨ False ) {∨ commutative } = y ∨ ( x ∧ False ) {∨ distributive } = y ∨ False {∧ null } = y {∨ identity } Figure 2: {∧ absorption } : ( x ∨ y ) ∧ y = y course starts in the domain of equations.We derive propositional logic from ten basic equations of Boolean algebra (Figure 1). The traditionaltruth tables, absorption equations, and so on are derived by reasoning from the basic equations in thestandard, algebraic way.This gives students practice, early on, in the syntax matching and step-by-step reasoning that is usedthroughout the course, but in a tightly prescribed context, where keeping track of formulas is relativelysimple. Figure 2 displays a typical proof that students would see or be asked to derive from the basicequations, or from other equations such as {∧ null } derived earlier from the basics.After some study of propositional logic, digital circuits are introduced as an alternate notation forBoolean formulas. We focus on and-, or-, and not-gates, but we also draw circuit diagrams withexclusive-or, nand, nor, and other standard gate symbols. We show, by reasoning from the basic Booleanequations, how the behavior of any circuit can be fully realized in terms of nand-gates alone. As anexercise, students show that an implication gate is universal in the same sense as nand.Gradually we work up to a ripple-carry adder circuit. All of the algebraic support for twos-complementarithmetic is defined and justified using standard equations of numeric algebra and, formally, by definingACL2 operators to carry out arithmetic on binary numerals. In this way, the students see a model in soft-ware of a ripple-carry adder, along with a diagram of a digital circuit for the adder at the gate level. Thefact that the operations the circuit performs are consistent with ordinary arithmetic on numbers is provedby induction, both informally (paper and pencil) and formally through ACL2. In addition, a softwaremodel for bignum addition and multiplication on binary numerals is developed, and its operations arejustified by informal and formal, mechanized proofs.This helps students understand how physical devices can perform computations, and that providesa basis for understanding how computers work at the circuit level. Students acquire an understanding2 HowComputers Work: Computational Thinking for Everyoneof how circuits do what they do, and how engineers can know, for certain, some of the operationalproperties of circuits, modulo, of course, failures in physical properties of the circuits. We define anisomorphism between circuit diagrams and formulas in logic and discuss what this isomorphism meansabout the connection between formal logic and physical devices. We claim that digital circuits are physi-cal representations of logic formulas and the two are, therefore, subject to the same sort of mathematicalanalysis.The same goes for the equation-based software discussed in the course. Stretching this to conven-tional software would be impractical at best, but not theoretically impossible, since conventional softwareis written in a formal language. To mention just one of the many problems in this area, the lack of formalsemantics for conventional programming languages presents obstacles that current research has barelybegun to breach. We discuss these issues in class, more in the honors course than in the version of thecourse for computer science students. However, students are not required to understand it deeply in eithercourse. Examinations and homework problems do not call for a mastery of these ideas. A course emphasizing the logical foundations of computing can leave students (computing majors, espe-cially, but other students, too) with the erroneous impression that none of this is relevant for real-worldcomputing. To address this situation, students are presented with a selection of real-world applicationsthat they are familiar with. The applications are chosen carefully so that they resonate with students andelaborate on principles that have already been discussed in class.One such application is Facebook. Students see a quick overview of the history of web applications,with a focus on the distinction between Web 1.0 and Web 2.0 applications. Both of these are dynamicweb applications, and students learn how the dynamics are performed. In particular, they see how awebpage consists of a template that can be “filled in” with information that may come from a productcatalog, for example. The key problem is one of searching ( i.e. , finding the relevant information todisplay on this web page), and that is what varies between Web 1.0 and Web 2.0 applications.What students learn is that Web 2.0 applications mix content from application producers with contentfrom consumers. Different consumers may see completely different results, due to customization andpersonalization. Facebook, the ultimate Web 2.0 application, actually provides very little content in thetraditional sense. Most of the content is contributed by each user’s circle of friends.Students also learn the concept (but not the working details) of relational databases and how theycan be used to generate content for Web 1.0 applications, such as traditional storefronts. Students seesimple SQL queries, although they do not learn how to write their own queries, nor how to interact witha databases. Instead, they see relational databases as a solution to the key problem of finding the relevantinformation to display. Students can write the program that finds this information in a list, and theysee relational databases as more complicated versions of this process—which is true, while being anunmitigated oversimplification. This reinforces one of the big ideas in the course, namely that differentdefinitional properties can produce the same results but at vastly different computational expense.This point is driven further. The students then learn the Achilles heel of relational databases, namelythat join operations on large tables are prohibitively expensive. As a case in point, we observe that the joinof the “friends” and “statuses” tables that could, in principle, support Facebook is infeasible—anotherbig idea in the course, that computation expense can make some devices feasible and others infeasible.So how does Facebook do it? Facebook developers solved this problem by building their own non-relational database called Cassandra [9]. At a high level, Cassandra acts like a simple key-value store,and the students have experience with this concept, having earlier studied AVL trees. Again, studentsexPageand Ruben Gamboa 13see the big idea that different (more efficient or scalable) solutions of the same principle can yield vastlydifferent computational expense. But Cassandra is more than just a key/value store. It features conceptssuch as consistent hashing, database sharding, data replication, and eventual consistency. Although thedetails of these features are very technical, at a high level of abstraction they are accessible, even tonon-technical students. For example, students learn about Cassandra’s famous ring architecture, andthey quickly grasp how this architecture enables both the massive throughput required by Facebook andreliability in case of system failure, which is certain to occur from time to time in such a large application.This discussion does not prepare students to understand how NoSQL databases such as Cassandra areimplemented, or even how to use them. Rather, it serves to convince students that ideas and techniquesthat they have already seen can be scaled up to build large, important applications such as Facebook.To give some idea of the depth of the material on Cassandra, it is discussed in one, 75-minute lecture,including questions from students and a short discussion period near the end. These discussions havebeen lively, which indicates that the majority of the students get enough out of the lecture to makeintelligent observations about the topic.
Another example, Google’s technology stack, reinforces the lesson that the basic computer principlesstudents have studied in the course have real-world implications. The students are, of course, familiarwith Google products, such as the search engine and Gmail, as paragons of web applications. Now weexpose them to a piece of Google’s technology—MapReduce [8]. This topic is discussed in a mannersimilar to the discussion of Cassandra (Section 4.7): at a high level only in one 75-minute lecture,including a lively class discussion.MapReduce is part of Google’s approach to distributed computing. Jobs are broken down into mapand reduce steps that operate on dictionaries (key/value data structures). A MapReduce program can bedeveloped entirely on a single computer. Then, the MapReduce framework takes care of the details ofexecuting individual map and reduce tasks on hundreds or thousands of computers.This particular technology is chosen because it is easily motivated by Google’s massive scaling needs.In this way students see, once more, how dealing with large scale is the main difference between com-puting concepts as they have experienced them and engineering as practiced in real-world computing.Another reason for choosing MapReduce is because of its roots in functional programming, which im-mediately ties into programming concepts that students have learned in the course.Although MapReduce was originally implemented in C++, its key ideas can be rendered in manyother languages, such as Java in the Apache Hadoop project [23]. In this class, students see someMapReduce operations expressed in the form of ACL2 functions. They see simple examples, such asdistributed word count and distributed grep—classic examples from the MapReduce literature.Students also see how the MapReduce framework can be used to perform meaningful work at scale,such as inverting the graph of internet links, which is then used to calculate PageRank [24] for Google’ssearch results.An important aspect of this presentation is that the (admittedly simple) programs students see arecomplete. A version of the MapReduce framework is implemented in ACL2, so students can see howthe map and reduce functions are combined to produce a complete solution. This solution is sequential,and it runs on just one machine. But students learn that the full MapReduce framework distributes thesetasks across hundreds, even thousands of computers. Again, students see the big idea that differentsets of definitional properties ( i.e. , the sequential and distributed versions of MapReduce) produce the4 HowComputers Work: Computational Thinking for Everyonesame results at vastly different computational expense, and that the difference is sufficient to make largeMapReduce applications (indexing the web, for example) feasible.This example solution to a problem of massive scale, another other of the big ideas in the course,together with the Cassandra solution to accessing massive, fast changing, data bases discussed in Section4.7, together provide some illumination of the massive scale idea.
One of the big ideas in the course is how important, complex algorithms derive from simple, definitionalproperties. The two most complex algorithms that we discuss in detail are merge-sort and AVL-treeinsertion. Yet, the approach to these algorithms is the same as for all other software artifacts we present.Namely, we assume someone has given us an operator that carries out the computation of interest.We look for properties that we expect the operator would have if it worked properly, and we try to find acollection of properties that are consistent, complete, and constructive, since such properties must definean operator. Unless we misconstrue our expectations, that operator will be the one we want to define.For example the merge portion of the merge-sort algorithm assumes that its arguments are lists whoseelements are arranged in order by increasing value. We divide the data space into three parts: (1) the firstargument is the empty list, (2) the first argument is not empty, but the second argument is, and (3) botharguments are non-empty. That covers all the possibilities, and none of the three parts of the data spaceoverlaps with another, so our equations will automatically be consistent.When either argument is empty, the result must be the other argument, so equations correspondingto these conditions are easy to write down. In case (3), when both arguments are non-empty, the firstelement of the merged list must be the smaller of the first elements in the lists supplied as arguments, andthe rest of the elements must be the merge of the remaining elements of list with the smaller first elementand all of the elements in the other list. Assuming the merge operator we were given works, that analysisyields the following equations.( merge ( cons x xs ) ( cons y ys )) = ( cons x ( merge xs ( cons y ys ))) if x ≤ y { mrg ≤} ( merge ( cons x xs ) ( cons y ys )) = ( cons y ( merge ( cons x xs ) ys )) if x > y { mrg > } As we noted earlier, the equations will be consistent because we have divided the data space intothree, non-overlapping parts. They are constructive because, in both of the inductive references to merge ,one of the arguments is a shorter list than the one we started with, which makes it closer to the corre-sponding argument on the left-hand side of a non-inductive equation.The merge-sort operator, which of course refers to the merge operator, is equally straightforwardto define. None of the definitions for merge-sort and its supporting operators is more than a few lineslong. The same is true of the AVL insertion code, given an appropriate breakdown of the problem. So,these complex computations derive from simple properties.Furthermore, we can use the equations, along with a model of basic, one-step operators such as cons , first , rest , selection (that is, if , once the Boolean condition has been computed), relationaloperators for numbers, and the like to derive recurrence equations for the number of steps in a merge-sortcomputation. We solve these equations by proposing a solution and proving it by induction, and we findthat merge-sort is an n log ( n ) computation. A similar derivation of equations for insertion-sort and ananalysis of the number of computational steps these equations lead to shows that insertion-sort is an n algorithm.We do not discuss in the course details in the implementation of functional languages that affect thecomputational efficiency of individual operations such as cons , first , and rest in our computationalmodel. Instead, we focus on another big idea in the course, namely that different definitional propertiesexPageand Ruben Gamboa 15can produce the same results at vastly different computational expense. A table comparing the growthof n log ( n ) with that of n shows just how infeasible the insertion-sort algorithm is for large data sets,another big idea in the course.Thus, all the big ideas listed in Section 1 are discussed in the course, but at widely different levelsof detail. There are, of course, many different ways to slice and dice the material to be covered. Forexample, one could choose to cover fewer algorithms and replace the omitted ones with more materialon interactive software. As it stands, we talk about interactive software early in the course and discuss amodel for it based on the methods employed in Racket software [12]. But, the students neither write anyinteractive software nor reason about it, so its presence in the course is a minor one. In summary, thisreport describes a particular set of choices about what details to include where, and with respect to whatbig ideas in a course on computational thinking for courses in which most students are not majoring incomputer science. This is not the first course that is designed to expose non-majors to computer science. There is anongoing effort by the College Board to promote a new introductory computer science course designedfor advanced high school students or first-year college students [25].
Computer Science: Principles (CSP), the new course initiative, stands in contrast to the College Board’s existing introductory computerscience course and Advanced Placement (AP) Computer Science exam. The big change is that the CSPcourse does not focus on programming. Instead, it spends more time on trying to develop key insightson the nature and especially the relevance of computing in the modern world.We share the College Board’s goals, and we consider our course to be in the spirit of the CSP ap-proach. However, there are significant differences between our approach and that of the existing officialpilots for CSP [26]. Like ours, all of these courses involve programming to some extent, but withoutdwelling on the programming details. However, most of these courses use scripting languages to fa-cilitate programming. This makes it possible for students to build sophisticated programs with a smallamount of effort. For instance, Python is a popular choice of language, and so are visual languages likeBYOB (aka SNAP!) and App Inventor [27, 28]. These languages allow students to build graphical andmobile applications like the ones they normally interact with.For our course we chose to use the language ACL2 instead of a scripting language because thetheorem prover that comes with ACL2 makes it easier for the students to reason about the programs theybuild. We think the introduction of methods of ensuring software correctness adds value to discussionsabout the relevance of computer software, so we chose to include material on testing and verification atthe expense of, for example, multimedia applications.Other courses aimed at the same audience, with “computational thinking” as an organizing theme,have been springing up at many universities [29]. Common threads in these courses include a briefintroduction to programming, societal impacts of computer science, and discussions on the limits ofcomputing. The big ideas in our honors course include most of these ideas, but our approach is a logic-based introduction to computational thinking.An examination of computing programs at fifteen large universities in the central United States sug-gests that about a third of such programs require a course in logic that is separate from the requireddiscrete math course. In many cases, the required logic course focuses on digital circuits, and thosecourses rarely include any material on reasoning about properties of circuits. However, three of thefifteen programs did offer a course on reasoning about circuits and/or software.6 HowComputers Work: Computational Thinking for EveryoneEvidence of the integration of mechanized logic into undergraduate courses is harder to find. Mano-lios uses ACL2, via the ACL2 Sedan [30], in a lower division logic course at Northeastern Universitythat was introduced in an experimental form by Felleisen [31]. The Manolios course, which is called“Logic and Computation,” is probably the work that relates most closely to this paper. Our reading of thecourse description is that it covers ACL2 in greater detail than our course, but contains less “big picture”material.Harper, Pfenning, and Erdmann have revised computer science courses at Carnegie Mellon Uni-versity to incorporate reasoning about software throughout the curriculum, right from the beginning.Erdmann’s notes [32] on a lower division course called “Principles of Programming” and Pfenning’snotes [33] on “Principles of Imperative Computation” provide some insight into how Carnegie Mellonis dramatically recasting computer science to have reasoning as a central element of computer program-ming.Tinelli [34] has included assignments in the use of KeY [35], a tool for theorems and proofs aboutprograms written in a subset of Java. Courses employing Haskell often use QuickCheck [14], whichgives students practice in stating properties as logic formulas, an important skill in using computationallogic systems. Jackson’s Alloy system is used in undergraduate classes and exposes students to logic asa tool for stating and verifying properties of software components [36].Our review of related work is not comprehensive. No doubt there are many efforts that we have notmanaged to find. The literature in this area is sparse, but we think the papers and projects discussed inthis section provide a context in which to consider the work reported in this paper.
A course including topics in classical logic, digital circuits, programming, testing, verification, and othermajor computing concepts, all of it couched in terms of a familiar form of reasoning, namely algebraicequations, can provide a comfortable, yet challenging environment for interested students, regardlessof background, assuming a standard, college-prep education that includes high-school algebra. Such acourse can provide a basis for understanding in a fundamental way what computers do and how they doit. It is one way to introduce students to computational thinking.This is not a “soft skills” course. It calls for careful thinking, and it rewards studious attention. Yet,it is accessible and interesting to a broad base of college students. This combination of depth, challenge,and reward offers students something new and valuable.An essential component of this enterprise is an equation-based programming language with a property-based testing facility. A mechanized logic with a quick-entry learning curve enhances the experience andthe educational impact relative to the learning effort that students invest. A conventional programminglanguage will not serve because it cannot be understood in terms of classical logic and the standardmechanisms of algebraic reasoning. The theme of equations cannot be carried throughout a course inwhich the programming component relies on the imperative paradigm.We are writing a textbook that takes the approach and includes the material discussed in this paper.Drafts of the text have been used to provide readings in the course. We plan to develop interactive, web-accessible learning tools, including mini-tutorials, exercises, and automated assessment of solutions forinstant feedback. Lecture notes, homework projects, and examinations are available upon request toeducators who want to incorporate some of these ideas into their work.The new course in the principles of computer science proposed by the College Board is designed forcollege preparatory students and first-year college students [25]. It emphasizes computational thinking.exPageand Ruben Gamboa 17The proposal elaborates seven big ideas and key concepts: creativity, abstraction, data, algorithms, pro-gramming, internet, and impact. Based on the descriptions in their proposal, we believe that the materialin our course and its accompanying text provide an effective learning environment for those ideas andconcepts. A course following this approach would be one way to introduce computational thinking to abroad range of students.
Acknowledgments.
The authors are grateful to the reviewers for their thoughtful and detailed comments and suggestions.Their ideas led to significant improvements in the paper, especially in the level of detail describingelements of the course, relating those to its big ideas, and in discussing the experiences that studentsreported in their assessments of the course.
References [1] R. Page (2009):
Computational Logic in the Undergraduate Curriculum . In S. Ray & D. Russinoff, editors:
Proceedings of the th International Workshop on the ACL2 Theorem Prover and Its Applications , pp. 29–32.[2] C. Eastlund (2009):
DoubleCheck Your Theorems . In S. Ray & D. Russinoff, editors:
Proceedings of the th International Workshop on the ACL2 Theorem Prover and its Applications , pp. 42–46.[3] R. Page, C. Eastlund, & M. Felleisen (2008):
Functional Programming and Theorem Proving for Undergrad-uates: A Progress Report . In F. Huch & A. Martin, editors:
Proceedings of the ACM SIGPLAN 2008 Workshopon Functional and Declarative Programming in Education , pp. 21–29, doi:10.1145/1411260.1411264.[4] D. Vaillancourt, R. Page, & M. Felleisen (2006):
ACL2 in DrScheme . In P. Manolios & M. Wilding, editors:
Proceedings of the th International Workshop on the ACL2 Theorem Prover and its Applications , pp. 107–116,doi:10.1145/1217975.1217999.[5] R. Page (2003):
Software Is Discrete Mathematics . In C. Runciman & O. Shivers, editors:
Proceedingsof the th International Conference on Functional Programming , SIGPLAN Notices 38 (3), pp. 79–86,doi:10.1145/944746.944713.[6] R. Page (2001):
Functional Programming ... and Where You Can Put It . In C. Norris & J. Fenwick Jr., editors:
ACM SIGPLAN Notices 36 (9), pp. 19–24, doi:10.1145/609769.609771.[7] R. Page (2007):
Engineering Software Correctness . J. Functional Programming 17 (6), pp. 675–686,doi:10.1017/S095679680700634X.[8] J. Dean & S. Ghemawat (2008):
MapReduce: Simplified Data Processing on Large Clusters . Commun. ACM51 (1), pp. 107-113, doi:10.1145/1327452.1327492.[9] A. Lakshman & P. Malik (2010):
Cassandra: A Decentralized Structured Storage System . ACM SIGOPSOperating Systems Review 44 (2), pp. 35–40, doi:10.1145/1773912.1773922.[10] J. McCarthy (1960):
Recursive functions of symbolic expressions and their computation by machine, Part I . Communications of the ACM 3 (4), pp. 184–195, doi:10.1145/367177.367199.[11] University of Oklahoma Honors Students (2011 & 2012): Student assessments of the course “How Comput-ers Work: Logic in Action.” Available at [12]
Racket Programming Language . Available at http://racket-lang.org/ [13] R. Findler, J. Clements, C. Flanagan, M. Flatt, S. Krishnamurthi, P. Steckler, & M. Felleisen (2002):
DrScheme: A Programming Environment for Scheme . J. Functional Programming 12 (2), pp. 159–182,doi:10.1017/S0956796801004208. [14] K. Claessen & J. Hughes (2000):
QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs .In M. Odersky & P. Wadler, editors:
Proceedings of the th ACM SIGPLAN International Conference onFunctional Programming, ACM SIGPLAN 35 (9), pp. 268–279, doi:10.1145/351240.351266.[15] C. Eggensperger (2012):
Proof Pad . Available at http://proofpad.org/ [16] M. Kaufmann, P. Manolios, & J.S. Moore (2000):
Computer-Aided Reasoning: An Approach . Kluwer Aca-demic Publishers.[17] M. Felleisen, R. Findler, M. Flatt & S. Krishnamurthi (2001):
How to Design Programs . MIT Press.[18] W. Hunt, R. Krug, & J. Moore (2003):
Linear and nonlinear arithmetic in ACL2 . In D. Geist & E. Tronci,editors:
Correct Hardware Design and Verification Methods, Proceedings of CHARME 2003 . Springer-Verlag.pp. 319–333, doi:10.1007/978-3-540-39724-3 29.[19] L. Paulson (1994):
Isabelle: A Generic Theorem Prover, LNCS 828 . Springer-Verlag,doi:10.1007/BFb0030541.[20] Y. Bertot & P. Cast`eran (2004):
Interactive Theorem Proving and Program Development, Coq’Art: TheCalculus of Inductive Constructions . Springer-Verlag, doi:10.1093/comjnl/bxh141.[21] A. Bove, P. Dybjer, & U. Norell (2009):
A Brief Overview of Agda – A Functional Language with De-pendent Types . In S. Berghofer, T. Nipkow, C. Urban, & M. Wenzel, editors:
Proceedings of the Interna-tional Conference on Theorem Proving in Higher Order Logics, LNCS 5674 . Springer-Verlag. pp. 73–78,doi:10.1007/978-3-642-03359-9 6.[22] R. Page (2011):
Property-Based Testing: A Catalog of Classroom Examples . In A. Gill & J. Hage, editors:
Proceedings of the 2011 Symposium on Implemenation and Application of Functional Languages, LNCS 7257 .Springer-Verlag. pp. 134–147, doi:10.1007/978-3-642-34407-7 9.[23] Apache Hadoop. http://hadoop.apache.org .[24] L. Page (2007):
Scoring Documents in a Linked Database . U.S. Patent 7,269,587.[25] The College Board (2011):
Computer Science: Principles . Available at [26] The College Board (2012):
CS Principles Pilot Sites . Available at [27] B. Harvey & J. M¨onig (2010):
Bringing “No Ceiling” to Scratch: Can One Language Serve Kids andComputer Scientists?
In:
Constructionism 2010 .[28] Y.C. Hsu, K. Rice, & L. Dawley (2012):
Empowering Educators with Google’s Android App Inventor: AnOnline Workshop in Mobile App Design . British Journal of Educational Technology 43 , pp. E1–E5,doi:10.1111/j.1467-8535.2011.01241.x.[29]
Center for Computational Thinking: Education . Available at [30] P. Dillinger, P. Manolios, D. Vroon, & J. Moore (2007):
ACL2s: “The ACL2 Sedan” . In: User Interfaces forTheorem Provers Workshop
Electronic Notes in Theoretical Computer Science 174 (2). Available at , Elsevier Science Publishers, pp. 3–18,doi:10.1016/j.entcs.2006.09.018.[31] C. Eastlund, D. Vaillancourt, & M. Felleisen (2007):
ACL2 for Freshman: First Experiences . In R. Gamboa,J. Sawada, & J. Cowles, editors:
Proceedings of the th International Workshop on the ACL2 Theorem Proverand Its Applications , pp. 200–211.[32] M. Erdmann (2011):
Notes on Principles of Programming . Available at [33] F. Pfenning (2011):
Notes on Principles of Imperative Computation . Available at [34] C. Tinelli (2008):
Formal Methods in Software Engineering . Available at exPageand Ruben Gamboa 19 [35] B. Beckert, R. H¨ahnle, & P. Schmitt, editors (2007):
Verification of Object-Oriented Software: the KeYapproach, LNCS 4334 , Springer-Verlag, doi:10.1007/978-3-540-69061-0.[36] D. Jackson (2012):