David Jackson

Grading student programs using ASSYST

The task of grading solutions to student programming exercises is laborious and error-prone. We have developed a software tool called ASSYST that is designed to relieve a tutor of much of the burden of assessing such programs. ASSYST offers a graphical interface that can be used to direct all aspects of the grading process, and it considers a wide range of criteria in its automatic assessment. Experience with the system has been encouraging.

A semi-automated approach to online assessment

Desirable though fully automated assessment of student programming assignments is, it is an area that is beset by difficulties. While it is not contested that some aspects of assessment can be performed much more efficiently and accurately by computer, there are many others that still require human involvement. We have therefore designed a system that combines the strengths of the two approaches, the assessment software calling upon the skills of the human tutor where necessary to make sensible judgements. The technique has been used successfully on a systems programming course for several years, and student feedback has been supportive.

Promoting phenotypic diversity in genetic programming

Population diversity is generally seen as playing a crucial role in the ability of evolutionary computation techniques to discover solutions. In genetic programming, diversity metrics are usually based on structural properties of individual program trees, but are also sometimes based on the spread of fitness values in the population. We explore the use of a further interpretation of diversity, in which differences are measured in terms of the behaviour of programs when executed. Although earlier work has shown that improving behavioural diversity in initial GP populations can have a marked beneficial effect on performance, further analysis reveals that lack of behavioural diversity is a problem throughout whole runs, even when other diversity levels are high. To address this, we enhance phenotypic diversity via modifications to the crossover operator, and show that this can lead to additional performance improvements.

Phenotypic diversity in initial genetic programming populations

A key factor in the success or otherwise of a genetic programming population in evolving towards a solution is the extent of diversity amongst its members. Diversity may be viewed in genotypic (structural) or in phenotypic (behavioural) terms, but the latter has received less attention. We propose a method for measuring phenotypic diversity in terms of the run-time behaviour of programs. We describe how this is applicable to a range of problem domains and show how the promotion of such diversity in initial genetic programming populations can have a substantial impact on solution-finding performance.

Layered learning in boolean GP problems

Layered learning is a decomposition and reuse technique that has proved to be effective in the evolutionary solution of difficult problems. Although previous work has integrated it with genetic programming (GP), much of the application of that research has been in relation to multi-agent systems. In extending this work, we have applied it to more conventional GP problems, specifically those involving Boolean logic. We have identified two approaches which, unlike previous methods, do not require prior understanding of a problems functional decomposition into sub-goals. Experimentation indicates that although one of the two approaches offers little advantage, the other leads to solution-finding performance significantly surpassing that of both conventional GP systems and those which incorporate automatically defined functions.

A software system for grading student computer programs

Abstract While there is a growing trend towards the exploitation of technology for the delivery of educational material, it must be said that techniques for computer-based assessment of students are still in their infancy. However, increasing numbers of students and the expanding popularity of distance learning make it not only desirable but imperative that technology be used to the full in measuring student progress. In the context of computer programming courses, assessment must include judgements of the quality of solutions to programming exercises. Although there is much that a computer cannot address in this regard, there are a number of aspects of assessment that can be automated to great effect. A software system has therefore been designed and implemented which employs a variety of techniques for grading student exercises in terms of program correctness, efficiency, style, complexity, and the adequacy of test data. For a given exercise, the system generates a report summarizing the results of each form of analysis and the way in which it contributes to the final grade. Experiences to date are encouraging, the system offering much faster and more comprehensive marking than was previously possible.

Using software tools to automate the assessment of student programs

Abstract Attempting to automate the process of assessing and grading student programs is not a frivolous exercise concerned merely with relieving tutors of this burden: the highly desirable goals of objectivity and accuracy of assessment form more serious considerations, while the advent of Computer Aided Instruction (CAI) systems for teaching introductory programming makes it imperative that such automation is attained. This paper examines the current status of approaches to automatic program assessment, and draws the lamentable conclusion that they are not yet sufficiently advanced to deal with the variety of program solutions that students tend to submit. It goes on to establish the pattern-matching and recognition capabilities that would be required of a system with the desired flexibility, and contends that the software technology for the provision of these facilities is already to hand in the form of compiler-writing tools. Several examples of typical student programming problems are considered, and the application of the Unix tools Lex and Yacc to the automatic assessment of these programs is described. Experience with this approach indicates that a comparatively small amount of effort is all that is necessary to generate automatic graders that are versatile and powerful enough to cope with a wide range of programming exercises.

Single node genetic programming on problems with side effects

Single Node Genetic Programming (SNGP) offers a new approach to GP in which every member of the population consists of just a single program node. Operands are formed from other members of the population, and evolution is driven by a hill-climbing approach using a single reversible operator. When the functions being used in the problem are free from side effects, it is possible to make use of a form of dynamic programming, which provides huge efficiency gains. In this research we turn our attention to the use of SNGP when the solution of problems relies on the presence of side effects. We demonstrate that SNGP can still be superior to conventional GP, and examine the role of evolutionary strategies in achieving this.

Fitness evaluation avoidance in Boolean GP problems

A technique has been devised which, via consideration of the program nodes executed during fitness evaluation, allows a genetic programming system to determine many instances in which invocation of the fitness function can be avoided. The nature of Boolean logic problems renders them of particular interest as a focus of study for the application of this technique, and experimental evidence shows that significant speed-ups in execution time can be achieved when evolving solutions to these problems.

A new, node-focused model for genetic programming

We introduce Single Node Genetic Programming (SNGP), a new graph-based model for genetic programming in which every individual in the population consists of a single program node. Function operands are other individuals, meaning that the graph structure is imposed externally on the population as a whole, rather than existing within its members. Evolution is via a hill-climbing mechanism using a single reversible operator. Experimental results indicate substantial improvements over conventional GP in terms of solution rates, efficiency and program sizes.