Mikko-Jussi Laakso

Exploring programming assessment instruments: a classification scheme for examination questions

This paper describes the development of a classification scheme that can be used to investigate the characteristics of introductory programming examinations. We describe the process of developing the scheme, explain its categories, and present a taste of the results of a pilot analysis of a set of CS1 exam papers. This study is part of a project that aims to investigate the nature and composition of formal examination instruments used in the summative assessment of introductory programming students, and the pedagogical intentions of the educators who construct these instruments.

Can computing academics assess the difficulty of programming examination questions

This paper reports the results of a study investigating how accurately programming teachers can judge the level of difficulty of questions in introductory programming examinations. Having compiled a set of 45 questions that had been used in introductory programming exams, we took three measures of difficulty for the questions: the expectations of the person who taught the course and set the exams; the consensus expectations of an independent group of programming teachers; and the actual performance of the students who sat the exams. Good correlations were found between all pairs of the three measures. The conclusion, which is not controversial but needed to be established, is that computing academics do have a fairly good idea of the difficulty of programming exam questions, even for a course that they did not teach. However, the discussion highlights some areas where the relationships show weaknesses.

On Study Habits on an Introductory Course on Programming.

Computer aided assessment systems enable the collection of exact time and date information on students’ activity on a course. These activity patterns reflect students’ study habits and these study habits further predict students’ likelihood to pass or fail a course. By identifying such patterns, those who design the courses can enforce positive study habits and to prevent or minimize habits that lead to poor student performance. Hypothetically, by identifying and adjusting the short-term patterns, the teachers might be able to do the same during the course. This publication examines students’ short-term study habits on an introductory level programming course and presents multiple statistically significant connections between students’ assignment submission patterns and their respective final grades. Students who receive the highest grade start and finish their work early, do not work on weekends, and do not work at night, whereas those who fail the course do not show similar behavior but exhibit significant enrichment among those who work large amounts during the night. Course’s mandatory tutorial sessions that act both as assignment release events and as collaborative assignment solving sessions strongly increase assignment submission counts regardless of the students’ final grades and ensure an early start to solving the assignments, possibly preventing those who would otherwise fail the course from starting their work near deadlines.

Redesigning an Object-Oriented Programming Course

Educational technology offers several potential benefits for programming education. Still, to facilitate the technology properly, integration into a course must be carefully designed. In this article, we present a redesign of an object-oriented university-level programming course. In the redesign, a collaborative education tool was utilized to enhance active learning, facilitate communication between students and teachers, and remodel the evaluation procedure by utilizing automatically assessed tasks. The redesign was based on the best practices found in our own earlier research and that of the research community, with a focus on facilitating active learning methods and student collaboration. The redesign was evaluated by comparing two instances of the redesigned course against two instances using the old methodology. The drop-out rate decreased statistically significantly in the redesigned course instances. Moreover, there was a trend toward higher grade averages in the redesigned instances. Based on the results, we can conclude that the utilization of educational technology has a highly positive effect on student performance. Still, making major changes to course methodology does not come without certain difficulties. Hence, we also present our experiences and suggestions for the course redesign to help other educators and researchers perform similar design changes.

Interaction Promotes Collaboration and Learning: Video Analysis of Algorithm Visualization Use during Collaborative Learning

We report a study on collaborative learning with Algorithm Visualizations (AV). We have previously confirmed the hypothesis that students’ higher engagement has a positive effect on learning outcomes when they learn collaboratively. Thus, in this paper the analysis is targeted on students’ collaborative learning process in order to find phenomena that explain the learning improvements. In the video and audio analysis of the learning sessions, we have identified that the amount of collaboration and discussion increase when the level of engagement increases. Furthermore, the groups that used visualizations on higher level of engagement, discussed the learned topic on different levels of abstraction whereas groups that used visualizations on lower levels of engagement tended to concentrate more on only one aspect of the topic. Therefore, one of our conclusions is that the level of engagement indicates, not only the learning performance, but also the amount of on-topic discussions in collaboration. Furthermore, based on previous literature, we claim that the amount and quality of discussions explain the learning performance differences when students use visualizations in collaboration on different levels of engagement.

Breaking the Programming Language Barrier: Using Program Visualizations to Transfer Programming Knowledge in One Programming Language to Another

The transition from one programming language to another is an issue, which usually needs to be addressed in programming curricula, as the learning is typically started with syntactically easier languages. This study explores the possibility to use a short interactive tutorial with visualization exercises to ease the transition from Python to Java. In the experiment, the students first took a pre-test to measure their earlier programming knowledge with Python. After that, they used the tutorial with visualization exercises for 45 minutes. The tutorial and the exercises were designed to underline the syntactical and structural differences between Python and Java. Finally, the students answered to post-test, which contained questions similar to pre-test, but in Java. The results indicate, that the students were able to obtain similar program comprehension skills in Java that they previously had with Python. Moreover, the students seem to think that using such tutorials is highly beneficial in the transition. Hence, we conclude, that ViLLE can be effectively used to ease the transition from one language to another.

Interactive Learning Content for Introductory Computer Science Course Using the ViLLE Exercise Framework

Interactive learning content is a key to the success of online learning materials. Hence, many systems for creating and delivering interactive content has been developed in academia. In practice, however, teachers have to choose the suitable system from the multitude of available choices, thus limiting themselves to the features of that system. Furthermore, educational researchers have data collected in different formats by different systems. In this paper, we describe how we worked around these issues when implementing a new introductory computer science course using the ViLLE learning environment. The new course consists of six main topics, each of which is covered on a lecture and an online tutorial. The tutorials contain textual material together with numerous automatically assessed exercises with visual and immediate feedback. As we developed the course material, we integrated two educational systems - JSAV for algorithm simulation exercises and js-parsons for program construction tasks - into the ViLLE environment in order to use them on the course. This was enabled by the use of the framework for creating exercises into the ViLLE learning environment. The framework aims at easing the inclusion of interactive exercise into the environment by exercise system developers. In the paper, we briefly describe the course, the ViLLE exercise framework and out technical solution, as well as student results from the course and feedback on the exercise types.

Relationship between text and action conceptions of programming: a phenomenographic and quantitative perspective

Phenomenographic research studies have identified different understandings of the concepts class and object by novice programmers. Aspects of understanding include a focus on artefacts of text, syntax and structure (text), as active agents in a program (action) and as models of an external reality (model). We explore the hypothesis that these aspects of conceptual understanding form a hierarchy in which mastery of the text aspect is a necessary precondition for understanding objects as active agents and the action aspect is a precondition for model understandings. We use empirical data from the final examination of an introductory programming course to test the relationship between the text and action aspects. Our findings do not support the hypothesis of a hierarchy but rather suggest that text and action understandings develop in parallel.

How does collaboration affect algorithm learning? A case study using TRAKLA2 algorithm visualization tool

Algorithm visualization tools can potentially be used to help novices to learn the abstract concepts of algorithms and data structures. Still, the methods of using such tools must be carefully considered. We conducted a study to find out if the TRAKLA2 algorithm visualization tool can be used to teach these concepts effectively. Moreover, we wanted to find out if there is a difference in learning when students use the tool in collaboration with another student. Students participated in a two-hour lab session, where half of them used the tool independently, and the other half in collaboration with a peer. We found out, that using the tool had a substantial effect on the learning outcome for both groups. However, the collaborative use did not have any additional effect on the learning.

Automatic assessment of program visualization exercises

ViLLE is a visualization tool for teaching programming to novice programmers. It has an extendable support for multiple programming languages which enables language-independent learning of programming. As a new feature, ViLLE supports automatically assessed exercises. The exercises can be easily integrated into a programming course by using the TRAKLA2 web environment.