Lauri Malmi

A Review of Generic Program Visualization Systems for Introductory Programming Education

This article is a survey of program visualization systems intended for teaching beginners about the runtime behavior of computer programs. Our focus is on generic systems that are capable of illustrating many kinds of programs and behaviors. We inclusively describe such systems from the last three decades and review findings from their empirical evaluations. A comparable review on the topic does not previously exist; ours is intended to serve as a reference for the creators, evaluators, and users of educational program visualization systems. Moreover, we revisit the issue of learner engagement which has been identified as a potentially key factor in the success of educational software visualization and summarize what little is known about engagement in the context of the generic program visualization systems for beginners that we have reviewed; a proposed refinement of the frameworks previously used by computing education researchers to rank types of learner engagement is a side product of this effort. Overall, our review illustrates that program visualization systems for beginners are often short-lived research prototypes that support the user-controlled viewing of program animations; a recent trend is to support more engaging modes of user interaction. The results of evaluations largely support the use of program visualization in introductory programming education, but research to date is insufficient for drawing more nuanced conclusions with respect to learner engagement. On the basis of our review, we identify interesting questions to answer for future research in relation to themes such as engagement, the authenticity of learning tasks, cognitive load, and the integration of program visualization into introductory programming pedagogy.

Characterizing research in computing education: a preliminary analysis of the literature

This paper presents a preliminary analysis of research papers in computing education. While previous analysis has explored what research is being done in computing education, this project explores how that research is being done. We present our classification system, then the results of applying it to the papers from all five years of ICER. We find that this subset of computing education research has more in common with research in information systems than with that in computer science or software engineering; and that the papers published at ICER generally appear to conform to the specified ICER requirements.

How do students solve parsons programming problems?: an analysis of interaction traces

The process of solving a programming assignment is generally invisible to the teacher. We only see the end result and maybe a few snapshots along the way. In order to investigate this process with regard to Parsons problems, we used an online environment for Parsons problems in Python to record a detailed trace of all the interaction during the solving session. In these assignments, learners are to correctly order and indent a given set of code fragments in order to build a functioning program that meets the set requirements. We collected data from students of two programming courses and among other analyses present a visualization of the solution path as an interactive graph that can be used to explore such patterns and anomalies as backtracking and loops in the solution. The results provide insights into students solving process for these types of problems and ideas on how to improve the assignment environment and its use in programming education.

Jype - a program visualization and programming exercise tool for Python

Based on research into learning programming and a review of program visualization research, we designed an educational software tool that aims to target students apparent fragile knowledge of elementary programming which manifests as difficulties in tracing and writing even simple programs. Most existing tools build on a single supporting technology and focus on one aspect of learning. For example, visualization tools support the development of a conceptual-level understanding of how programs work, and automatic assessment tools give feedback on submitted tasks. We implemented a combined tool that closely integrates programming tasks with visualizations of program execution and thus lets students practice writing code and more easily transition to visually tracing it in order to locate programming errors. In this paper we present Jype, a web-based tool that provides an environment for visualizing the line-by-line execution of Python programs and for solving programming exercises with support for immediate automatic feedback and an integrated visual debugger. Moreover, the debugger allows stepping back in the visualization of the execution as if executing in reverse. Jype is built for Python, when most research in programming education support tools revolves around Java.

Theoretical underpinnings of computing education research: what is the evidence?

We analyze the Computing Education Research (CER) literature to discover what theories, conceptual models and frameworks recent CER builds on. This gives rise to a broad understanding of the theoretical basis of CER that is useful for researchers working in that area, and has the potential to help CER develop its own identity as an independent field of study.n Our analysis takes in seven years of publications (2005-2011, 308 papers) in three venues that publish long research papers in computing education: the journals ACM Transactions of Computing Education (TOCE) and Computer Science Education (CSEd), and the conference International Computing Education Research Workshop (ICER). We looked at the theoretical background works that are used or extended in the papers, not just referred to when describing related work. These background works include theories, conceptual models and frameworks. For each background work we tried to identify the discipline from which it originates, to gain an understanding of how CER relates to its neighboring fields. We also identified theoretical works originating within CER itself, showing that the field is building on its own theoretical works.n Our main findings are that there is a great richness of work on which recent CER papers build; there are no prevailing theoretical or technical works that are broadly applied across CER; about half the analyzed papers build on no previous theoretical work, but a considerable share of these are building their own theoretical constructions. We discuss the significance of these findings for the whole field and conclude with some recommendations.

Increasing Adoption of Smart Learning Content for Computer Science Education

Computer science educators are increasingly using interactive learning content to enrich and enhance the pedagogy of their courses. A plethora of such learning content, specifically designed for computer science education, such as visualization, simulation, and web-based environments for learning programming, are now available for various courses. We call such content smart learning content. However, such learning content is seldom used outside its host site despite the benefits it could offer to learners everywhere. In this paper, we investigate the factors that impede dissemination of such content among the wider computer science education community. To accomplish this we surveyed educators, existing tools and recent research literature to identify the current state of the art and analyzed the characteristics of a large number of smart learning content examples along canonical dimensions. In our analysis we focused on examining the technical issues that must be resolved to support finding, integrating and customizing smart learning content in computer science courses. Finally, we propose a new architecture for hosting, integrating and disseminating smart learning content and discuss how it could be implemented based on existing protocols and standards.

Have we missed something?: identifying missing types of research in computing education

In this paper, we introduce a new way to categorise existing educational research making it possible to find new previously overlooked research topics. This novel categorisation system is based on the didactic foci of the research papers. Our categorisation scheme is not data driven as in previously published categorisation systems but is derived from the didactic triangle, which is a theoretical model describing the elements of a teaching-studying-learning processes.n The didactic-focus-based categorisation system can be used to promote discussion about missing types of research foci within the computing education research (CER) community. In addition, the new categorisation system supports meta-level analysis of published research papers and thus contributes to the discussion of the goals and the present state of CER. We analyse previously existing categorisation systems and describe how our system differs. Finally, we give two examples how to apply the new theoretical categorisation system. First, we use research papers published in ICER conferences 2005-2009 as our source material to illustrate how to apply the new theoretical categorisation system for revealing a number of areas for novel research such that seem to have received little attention from the CER community. The second example highlights how the categorisation system can be used to find overlooked research topics on some specific research area (in our example students success in CS1).

Increasing Students’ Awareness of Their Behavior in Online Learning Environments with Visualizations and Achievement Badges

In online learning environments where automatic assessment is used, students often resort to harmful study practices such as procrastination and trial-and-error. In this paper, we study two teaching interventions that were designed to address these issues in a university-level computer science course. In the first intervention, we used achievement badges, a form of gamification, to reward students for submitting early, solving exercises with few iterations, and completing the exercises with full points. In the second intervention, we used heatmap visualizations that show a prediction of the students success if he or she continues to behave in the same way, based on data from earlier courses. The results from a controlled experiment show that the heatmap visualizations had an impact on how early the students submitted their exercises and how many points they earned from them. We also observed that performance approach goal orientation correlated with an interest towards the achievement badges whereas performance avoidance correlated with an interest towards the visualizations. The results indicate that increasing students awareness of their behavior can have a positive impact on their study practices and results. However, the same type of feedback may not suit all students because of individual differences.

Students' Ways of Experiencing Visual Program Simulation

Abstract Visual program simulation (VPS) is a new, interactive form of educational program visualisation, in which learners use graphical controls to direct a program’s execution rather than simply watching an animation of it. In this article, we report on a qualitative empirical study of novice programmers learning through VPS. From an analysis of interviews with learners and supplementary observational data, six different ways of perceiving learning through VPS emerged. While a rich understanding of VPS opens up rich possibilities for learning programming, we also uncovered several ways of understanding VPS that do not afford such opportunities. For instance, learners do not always attribute meaning to the visual elements, and even when they do, they may fail to associate the visualised concepts with programming practice. Our results concretise how learners have to experience a number of key insights about VPS before effective learning through VPS can take place. Based on our findings, we make pedagogical recommendations with the overall goal of improving the integration of VPS and the rest of the teaching and learning environment. Our results may find applications beyond VPS within educational program visualisation in general.

Automatic recognition of students' sorting algorithm implementations in a data structures and algorithms course

Computing educators often rely on black-box analysis to assess students work automatically and give feedback. This approach does not allow analyzing the quality of programs and checking if they implement the required algorithm. We introduce an instrument for recognizing and classifying algorithms (Aari) in terms of white-box testing to identify authentic students sorting algorithm implementations in a data structures and algorithms course. Aari uses machine learning techniques to classify new instances. The students were asked to submit a program to sort an array of integers in two rounds: at the beginning of the course before sorting algorithms were introduced, and after taking a lecture on sorting algorithms. We evaluated the performance of Aari with the implementations of each round separately. The results show that the sorting algorithms, which Aari has been trained to recognize, are recognized with an average accuracy of about 90%. When considering all the submitted sorting algorithm implementations (including the variations of the standard algorithms), Aari achieved an overall accuracy of 71% and 81% for the first and second round, respectively.n In addition, we analyzed the students implementations manually to gain a better understanding of the reasons of failure in the recognition process. This analysis revealed that students have many misconceptions related to sorting algorithms, which results in problematic implementations that are more inefficient compared with those of standard algorithms. We discuss these variations along with the application of the tool in an educational context, its limitations and some directions for future work.