Jeremy V. Ernst

Instructional Design + Knowledge Components: A Systematic Method for Refining Instruction

This paper reports on a systematic method used to improve an existing unit of instruction. The method is distinctive in combining steps of instructional design with knowledge components from a cognitively-based framework of learning. Instructional design is used to develop assessment instruments that incorporate information about student misconceptions. The method uses the assessment instruments to evaluate student performance and learning gains, while statistical analysis evaluates the quality of the instruments themselves using measures of difficulty and discrimination. Fine-grain insight into possible improvements is enabled by the knowledge components implicated by the assessment. The method is illustrated and evaluated by applying it to a unit of instruction on collection-based iteration in a computational thinking class. Data gathered during this evaluation highlights a number of opportunities within the unit to refine the instruction.

Towards a Concept Inventory for Algorithm Analysis Topics

We present initial results from our work towards developing a concept inventory for algorithm analysis (AACI) at the post-CS2 level. We used a Delphi process to identify a list of algorithm analysis topics that were considered both important and hard by surveying a panel of experienced instructors. Through a similar survey process, we identified a list of student misconceptions related to the identified topics. Based on this, a set of pilot AACI items were developed. We validated the misconceptions list by analyzing student responses to four administrations of the pilot AACI in two different universities during Fall 2015 and Spring 2016. Results revealed that a sufficient number of students held most of the misconceptions identified in the list.

A basic recursion concept inventory

Recursion is both an important and a difficult topic for introductory Computer Science students. Students often develop misconceptions about the topic that need to be diagnosed and corrected. In this paper, we report on our initial attempts to develop a concept inventory that measures student misconceptions on basic recursion topics. We present a collection of misconceptions and difficulties encountered by students when learning introductory recursion as presented in a typical CS2 course. Based on this collection, a draft concept inventory in the form of a series of questions was developed and evaluated, with the question rubric tagged to the list of misconceptions and difficulties.

Misconception-Driven Feedback: Results from an Experimental Study

The feedback given to novice programmers can be substantially improved by delivering advice focused on learners cognitive misconceptions contextualized to the instruction. Building on this idea, we present Misconception-Driven Feedback (MDF); MDF uses a cognitive student model and program analysis to detect mistakes and uncover underlying misconceptions. To evaluate the impact of MDF on student learning, we performed a quasi-experimental study of novice programmers that compares conventional run-time and output check feedback against MDF over three semesters. Inferential statistics indicates MDF supports significantly accelerated acquisition of conceptual knowledge and practical programming skills. Additionally, we present descriptive analysis from the study indicating the MDF student model allows for complex analysis of student mistakes and misconceptions that can suggest improvements to the feedback, the instruction, and to specific students.

Supporting students with disabilities and limited English proficiency: STEM educator professional development participation and perceived utility

Background: Professional development offerings assist K–12 educators in addressing new and evolving classroom dynamics, circumstances, and situations. With the emerging demands of an increasingly science, technology, engineering, and mathematics (STEM)-infused society, teachers are challenged to provide high-quality service and equitable educational opportunities to all STEM education students, particularly to those students who traditionally are underrepresented in comparison to their peers in STEM education and/or have aspirations of participation in STEM-related careers. This study investigated K–12 STEM educator participation and perceived utility regarding professional development addressing specific needs of students with identified categorical disabilities and limited English proficiency (LEP). Results: Collection and analysis methods employed data retrieval and tabulation from the 2011–2012 School and Staffing Survey (SASS) Teacher Questionnaire (TQ). The national restricted access dataset was used to identify targeted teacher populations as well as provide a profile of STEM teacher participation in practice-oriented professional development activities regarding the two specified student groups. The results were categorically summarized and compared across science, technology, and mathematics (STM) disciplines and also between STM educators, non-STM educators, and educators in general. Conclusions: The results indicated that STM teachers tended to engage in fewer professional development opportunities and dedicated fewer hours in the professional development regarding students with categorical disabilities and LEP than the remainder of the teaching population. Overall, STM teachers’ perceived utility of the provided professional development experience was lower than that of the remainder of the teaching population.