Petri Ihantola

Review of recent systems for automatic assessment of programming assignments

This paper presents a systematic literature review of the recent (2006--2010) development of automatic assessment tools for programming exercises. We discuss the major features that the tools support and the different approaches they are using both from the pedagogical and the technical point of view. Examples of these features are ways for the teacher to define tests, resubmission policies, security issues, and so forth. We have also identified a list of novel features, like assessing web software, that are likely to get more research attention in the future. As a conclusion, we state that too many new systems are developed, but also acknowledge the current reasons for the phenomenon. As one solution we encourage opening up the existing systems and joining efforts on developing those further. Selected systems from our survey are briefly described in Appendix A.

Taxonomy of effortless creation of algorithm visualizations

The idea of using visualization technology to enhance the understanding of abstract concepts, like data structures and algorithms, has become widely accepted. Several attempts have been made to introduce a system that levels out the burden of creating new visualizations. However, one of the main obstacles to fully taking advantage of algorithm visualization seems to be the time and effort required to design, integrate and maintain the visualizations.Effortlessness in the context of algorithm visualization is a highly subjective matter including many factors. Thus, we first introduce a taxonomy to characterize effortlessness in algorithm visualization systems. We have identified three main categories based on a survey conducted among CS educators: i) scope, i.e. how wide is the context one can apply the system to ii) integrability, i.e., how easy it is to take in use by a third party, and iii) interaction techniques, i.e., how well does the system support different use cases regularly applied by educators. We will conclude that generic and effortless visualization systems are needed. Such a system, however, needs to combine a range of characteristics implemented in many current AV systems.

Eye tracking in computing education

The methodology of eye tracking has been gradually making its way into various fields of science, assisted by the diminishing cost of the associated technology. In an international collaboration to open up the prospect of eye movement research for programming educators, we present a case study on program comprehension and preliminary analyses together with some useful tools. The main contributions of this paper are (1) an introduction to eye tracking to study programmers; (2) an approach that can help elucidate how novices learn to read and understand programs and to identify improvements to teaching and tools; (3) a consideration of data analysis methods and challenges, along with tools to address them; and (4) some larger computing education questions that can be addressed (or revisited) in the context of eye tracking.

How (not) to introduce badges to online exercises

Achievement badges are increasingly used to enhance educational systems and they have been shown to affect student behavior in different ways. However, details on best practices and effective concepts to implement badges from a non-technical point of view are scarce. We implemented badges to our learning management system, used them on a large course and collected feedback from students. Based on our experiences, we present recommendations to other educators that plan on using badges.

Increasing Adoption of Smart Learning Content for Computer Science Education

Computer science educators are increasingly using interactive learning content to enrich and enhance the pedagogy of their courses. A plethora of such learning content, specifically designed for computer science education, such as visualization, simulation, and web-based environments for learning programming, are now available for various courses. We call such content smart learning content. However, such learning content is seldom used outside its host site despite the benefits it could offer to learners everywhere. In this paper, we investigate the factors that impede dissemination of such content among the wider computer science education community. To accomplish this we surveyed educators, existing tools and recent research literature to identify the current state of the art and analyzed the characteristics of a large number of smart learning content examples along canonical dimensions. In our analysis we focused on examining the technical issues that must be resolved to support finding, integrating and customizing smart learning content in computer science courses. Finally, we propose a new architecture for hosting, integrating and disseminating smart learning content and discuss how it could be implemented based on existing protocols and standards.

Service-Oriented Approach to Improve Interoperability of E-Learning Systems

We present a design and open source implementation for a service oriented e-learning system, which utilizes external services for supporting a wide range of learning content and also offers a REST API for external clients to fetch information stored in the system. The design will separate different concerns, such as user authentication and exercise assessment, into separate services, which together form a complete e-learning environment. A key component of the design is identifying a general set of characteristics among existing exercise assessment systems, by which the assessment methods are grouped into three types: synchronous, asynchronous and static exercises.

Two-Dimensional Parson's Puzzles: The Concept, Tools, and First Observations

Parson’s programming puzzles are a family of code construction assignments where lines of code are given, and the task is to form the solution by sorting and possibly selecting the correct code lines. We introduce a novel family of Parson’s puzzles where the lines of code need to be sorted in two dimensions. The vertical dimension is used to order the lines, whereas the horizontal dimension is used to change control flow and code blocks based on indentation as in Python. Python blocks have no explicit begin/end statements or curly braces to mark where the block starts or stops. Instead, indentation is used to define starts and stops of blocks and functions.

A mobile learning application for parsons problems with automatic feedback

In this paper, we present a tool that facilitates the learning of programming by providing a mobile application for Parsons problems. These are small assignments where learners build programs by ordering and indenting fragments of code. Parsons problems are well-suited to the mobile context as the assignments form small chunks of learning content that individually require little time to go through and may be freely divided across multiple learning sessions. Furthermore, in response to previous analysis of students using a web environment for Parsons problems, we describe improvements to the automatic feedback given in these assignments.

Adapting moodle to better support CS education

Many commercial or open-source systems for organizing courses are available, offering access to course materials, communication support, and receiving and grading student submissions. However, most of these systems are by default not ideally prepared to address specific demands of Computer Science (CS) education. We explore how Moodle as one of the most popular and free systems can be better adapted to support the needs of CS education and provide concrete guidance on features and extensions that could be explored. This report and work based on it can significantly improve courses for educators and students alike

How novices tackle their first lines of code in an IDE: analysis of programming session traces

While computing educators have put plenty of effort into researching and developing programming environments that make it easier for students to create their first programs, these tools often have only little resemblance with the tools used in the industry. We report on a study, where students with no previous programming experience started to program directly using an industry strength programming environment. The programming environment was augmented with logging capability that recorded every keystroke and event within the system, which provided a view on how the novices tackle their first lines of code. Our results show that while at first, the students struggle with syntax -- as is typical with learning a new language -- no evidence can be found that suggests that learning to use the programming environment is hard. In a two-week period, the students learned to use the basic features of the programming environment such as specific shortcuts. Although we observed students using copy-paste-programming relatively often, most of the pasted code is from their own previous work. Finally, when considering the compilation errors and error distributions, we hypothesize that the errors are a product of three factors; the exercises, the environment, and the data logging granularity.