Brian Dorn

Measuring cognitive load in introductory CS: adaptation of an instrument

A students capacity to learn a concept is directly related to how much cognitive load is used to comprehend the material. The central problem identified by Cognitive Load Theory is that learning is impaired when the total amount of processing requirements exceeds the limited capacity of working memory. Instruction can impose three different types of cognitive load on a students working memory: intrinsic load, extraneous load, and germane load. Since working memory is a fixed size, instructional material should be designed to minimize the extraneous and intrinsic loads in order to increase the amount of memory available for the germane load. This will improve learning. To effectively design instruction to minimize cognitive load we must be able to measure the specific load components for any pedagogical intervention. This paper reports on a study that adapts a previously developed instrument to measure cognitive load. We report on the adaptation of the instrument to a new discipline, introductory computer science, and the results of measuring the cognitive load factors of specific lectures. We discuss the implications for the ability to measure specific cognitive load components and use of the tool in future studies.

Aggregate Compilation Behavior: Findings and Implications from 27,698 Users

The error quotient (EQ) was first reported in 2006 as a behavioral measure of novice programmers. The EQ scores how well students deal with correcting syntax errors (or not) in their programs. The original studies were carried out on data collected using BlueJ, a pedagogic Java programming environment; today, newly installed instances of BlueJ capture data similar to these early studies automatically, meaning data regarding nearly 2 million programmers is captured every year by the Blackbox project. In this paper, we apply Jaduds original error quotient algorithm to this new, massive data set, and discuss our results and analysis in light of related work. Further, we consider the implications of our findings for researchers and educators in applying the EQ to 27,698 users in 10 different countries during the fall term of 2013.

Piloting TrACE: Exploring Spatiotemporal Anchored Collaboration in Asynchronous Learning

The use of multimedia content such as video is becoming more prevalent in educational environments. However, current platforms for hosting these media provide few collaborative tools to foster social learning between students or request help from instructors. In this paper, we explore the potential of spatiotemporal anchored collaboration, and we present a prototype media-playback environment called TrACE that exemplifies the approach. We examine a first design-based research (DBR) pilot deployment of TrACE in two post-secondary courses. Results indicate that students do take advantage of the systems affordances to interact in meaningful ways, though overall student annotation authoring was limited. Using the pilot data, we propose socio-technical modifications for the next iteration in the DBR cycle. Specifically we focus on tools to support instructors use of the system and for promoting collaboration between students.

Empirical validation and application of the computing attitudes survey

Student attitudes play an important role in shaping learning experiences. However, few validated instruments exist for measuring student attitude development in a discipline-specific way. In this paper, we present the design, development, and validation of the computing attitudes survey (CAS). The CAS is an extension of the Colorado Learning Attitudes about Science Survey and measures novice to expert attitude shifts about the nature of knowledge and problem solving in computer science. Factor analysis with a large, multi-institutional data-set identified and confirmed five subscales on the CAS related to different facets of attitudes measured on the survey. We then used the CAS in a pre–post format to demonstrate its usefulness in studying attitude shifts during CS1 courses and its responsiveness to varying instructional conditions. The most recent version of the CAS is provided in its entirety along with a discussion of the conditions under which its validity has been demonstrated.

An Analysis of HTML and CSS Syntax Errors in a Web Development Course

Many people are first exposed to code through web development, yet little is known about the barriers beginners face in these formative experiences. In this article, we describe a study of undergraduate students enrolled in an introductory web development course taken by both computing majors and general education students. Using data collected during the initial weeks of the course, we investigate the nature of the syntax errors they make when learning HTML and CSS, and how they resolve them. This is accomplished through the deployment of openHTML, a lightweight web-based code editor that logs user activity. Our analysis reveals that nearly all students made syntax errors that remained unresolved in their assessments, and that these errors continued weeks into the course. Approximately 20% of these errors related to the relatively complex system of rules that dictates when it is valid for HTML elements to be nested in one another. On the other hand, 35% of errors related to the relatively simple tag syntax determining how HTML elements are nested. We also find that validation played a key role in resolving errors: While the majority of unresolved errors were present in untested code, nearly all of the errors that were detected through validation were eventually corrected. We conclude with a discussion of our findings and their implications for computing education.

The Case for Validated Tools in Computer Science Education Research

Measuring student learning in computer science education presents particular challenges, making objective assessment elusive to educators and researchers. Development and validation of the FCS1 and CAS assessment tools demonstrate both the feasibility and benefits of validated assessment instruments in this discipline.

Computing on the Silicon Prairie: The State of CS in Nebraska Public Schools

Access to high quality K-12 computer science education has garnered considerable public attention in the United States in the wake of Code.orgs December 2013 Hour of Code. An increasing number of states are now counting CS coursework towards high school graduation requirements, and teacher certification for computing courses is being revisited across the country. In this work, we explore the current state of computer science education in Nebraska public schools using objective enrollment data from the 2013-2014 academic year. We find limited access to introductory programming courses, while basic computer/IT literacy coursework is commonplace. Lack of CS courses is particularly acute in small schools, which make up a majority of Nebraskas public schools. We also present data illustrating limited pursuit of IT teaching endorsement/certification in the state and highlight current issues for teacher training programs that lead to such certification in Nebraska.

Lost while searching: difficulties in information seeking among end-user programmers

End-user programmers, those who write code but lack formal training in computer science, are often reliant on various tools such as API documentation or searching the Web for information in order to complete a specific task. This study examines the information foraging behaviors of a group of web and graphic designers engaged in a series of code modification tasks. We find that users were largely unsuccessful in their foraging activities, with few information seeking events resulting in noticeable changes to participants source code. Participants viewed remarkably few results generated by their queries and rarely refined queries multiple times. However, these kinds of activities were positively correlated with task success metrics. We conclude with a discussion of the studys results and their implications on the design of future programming environments and search tools for end-user programmers.

Identification of the Emergent Leaders within a CSE Professional Development Program

The need for high quality, sustainable Computer Science Education (CSE) professional development (PD) at the grades K-12 level is essential to the success of the global CSE initiatives. This study investigates the use of Social Network Analysis (SNA) to identify emergent teacher leaders within a high quality CSE PD program. The CSE PD program was designed and implemented through collaboration between the computer science and teacher education units at a Midwestern metropolitan university in North America. A unique feature of this specific program is in the intentional development of a social network. This study discusses the importance of social networks, the development of social capital, and its impact on the sustainability of the goals of the CSE PD program. The role of emergent teacher leaders in the development of the social capital of the CSE PD cohort is investigated using SNA techniques. The cohort consisted of 16 in-service teachers in grades 6-12 representing seven districts and four distinct content areas. The instruments used involved a questionnaire and the results of a CSE PD program online course. The findings suggest a correlation between the emergent teacher leaders, the online course results, and the overall cohort social capital. Future uses of SNA within professional development programs are also discussed.

Thanks for Sharing: CS Pedagogical Content Knowledge Sharing in Online Environments

Programs such as CS10K and CS for All in the United States seek to increase the number of CS teachers and provide training to them, highlighting the need to understand how best to support teachers at disadvantaged schools. Compared with teachers at larger or more affluent schools, they are less likely to gain CS pedagogical content knowledge (PCK) through methods such as face to face conversations, attending conferences, or observing a colleagues class. However CS PCK is necessary for successful teaching and learning, and research has shown that teachers want opportunities to engage in reflective practice with peers. Thus teachers at underserved or isolated schools must rely on other methods to discuss and share CS PCK. We explore this problem by examining the current landscape of online resources used for this purpose. Data was collected from two online communication fora---one with lightly moderated content devoted to general CS education matters, and one with highly curated content devoted entirely to CS PCK. We analyze the strengths and weaknesses of both fora, and synthesize our results by examining design recommendations for an online CS PCK resource that would capture the benefits of both environments.