Richard Gluga

Foundations for Modeling University Curricula in Terms of Multiple Learning Goal Sets

It is important, but very challenging, to design degree programs, so that the sequence of learning activities, topics, and assessments over three to five years give an effective progression in learning of generic skills, discipline-specific learning goals and accreditation competencies. Our CUSP (Course and Unit of Study Portal) system tackles this challenge, by helping subject teachers define the curriculum of their subject, linking it to Faculty and institutional goals. The same information is available to students, enabling them to see how each subject relates to those goals. It then gives additional big-picture views of the degree for the academics responsible for the whole degree, including the ability to easily assess if a degree meets accreditation requirements. CUSP achieves this by exploiting a lightweight semantic mapping approach that gives a highly flexible and scalable way to map learning goals from multiple internal and external accrediting sources across the degree. We report its validation as used in a live university environment, across three diverse faculties, with 277 degrees and 7,810 subject sessions over a period of three years. Data from this evaluation indicates steady improvement in the documentation of the relationships between subjects, assessments, learning outcomes, and program level goals. This is driven by the reporting tools and visualizations provided by CUSP, which enable program designers and lecturers to identify parts of the curriculum that are unclear. This improved documentation of the curriculum enables more accurate and immediate quality reviews. Key contributions of this work are: a validated new approach for curriculum design that helps address the complexity of ensuring learners progressively develop generic skills; and a validated lightweight semantic mapping approach that can flexibly support visualizing the curriculum against multiple sets of learning goal frameworks.

Over-confidence and confusion in using bloom for programming fundamentals assessment

A computer science student is required to progress from a novice programmer to a proficient developer through the programming fundamentals sequence of subjects. This paper deals with the capturing and representation of learning progression. The key contribution is a web-based interactive tutorial that enables computer science educators to practice applying the Bloom Taxonomy in classifying programming exam questions. The tutorial captures participant confidence and self-explanations for each Bloom [5] classification exercise. The results of an evaluation with ten participants were analyzed for consistency and accuracy in the application of Bloom. The confidence and self-explanation measures were used to identify problem areas in the application of Bloom to programming fundamentals. The tutorial and findings are valuable contributions to future ACM/IEEE CS curriculum revisions, which are expected to have a continued emphasis on Bloom [1].

Modeling long term learning of generic skills

Many of the most important learning goals can only be achieved over several years. Our CUSP system helps achieve this over the 3-to-5 years of a university degree: it enables each teacher to map their own subject design to institutional learning goals; it creates both subject and degree-level models. It tackles the semantic mapping challenges using a highly flexible lightweight approach. We report its validation for 102 degrees and 1237 subject sessions. CUSP makes a contribution to understanding how to model long term learning of generic skills, using a lightweight semantic mapping based on multiple sets of externally defined learning goals. The work contributes to understanding of how to create comprehensive models of long term learning within degrees that are practical in real environments.

Mastering cognitive development theory in computer science education

To design an effective computer science curriculum, educators require a systematic method of classifying the difficulty level of learning activities and assessment tasks. This is important for curriculum design and implementation and for communication between educators. Different educators must be able to use the method consistently, so that classified activities and assessments are comparable across the subjects of a degree, and, ideally, comparable across institutions. One widespread approach to supporting this is to write learning objects in terms of Bloom’s Taxonomy. This, or other such classifications, is likely to be more effective if educators can use them consistently, in the way experts would use them. To this end, we present the design and evaluation of our online interactive web-based tutorial system, which can be configured and used to offer training in different classification schemes. We report on results from three evaluations. First, 17 computer science educators complete a tutorial on using Bloom’s Taxonomy to classify programming examination questions. Second, 20 computer science educators complete a Neo-Piagetian tutorial. Third evaluation was a comparison of inter-rater reliability scores of computer science educators classifying programming questions using Bloom’s Taxonomy, before and after taking our tutorial. Based on the results from these evaluations, we discuss the effectiveness of our tutorial system design for teaching computer science educators how to systematically and consistently classify programming examination questions. We also discuss the suitability of Bloom’s Taxonomy and Neo-Piagetian theory for achieving this goal. The Bloom’s and Neo-Piagetian tutorials are made available as a community resource. The contributions of this paper are the following: the tutorial system for learning classification schemes for the purpose of coding the difficulty of computing learning materials; its evaluation; new insights into the consistency that computing educators can achieve using Bloom; and first insights into the use of Neo-Piagetian theory by a group of classifiers.

On the reliability of classifying programming tasks using a neo-piagetian theory of cognitive development

Recent research has proposed Neo-Piagetian theory as a useful way of describing the cognitive development of novice programmers. Neo-Piagetian theory may also be a useful way to classify materials used in learning and assessment. If Neo-Piagetian coding of learning resources is to be useful then it is important that practitioners can learn it and apply it reliably. We describe the design of an interactive web-based tutorial for Neo-Piagetian categorization of assessment tasks. We also report an evaluation of the tutorials effectiveness, in which twenty computer science educators participated. The average classification accuracy of the participants on each of the three Neo-Piagetian stages were 85%, 71% and 78%. Participants also rated their agreement with the expert classifications, and indicated high agreement (91%, 83% and 91% across the three Neo-Piagetian stages). Self-rated confidence in applying Neo-Piagetian theory to classifying programming questions before and after the tutorial were 29% and 75% respectively. Our key contribution is the demonstration of the feasibility of the Neo-Piagetian approach to classifying assessment materials, by demonstrating that it is learnable and can be applied reliably by a group of educators. Our tutorial is freely available as a community resource.

An architecture for systematic tracking of skills and competence level progression in computer science

A typical Computer Science degree is three to five years long, consists of four to six subjects per semester, and two semesters per year. A student enrolled in such a degree is expected to learn both discipline-specific skills and transferable generic skills. These skills are to be taught in a progressive sequence through the duration of the degree. As the student progresses through the subjects and semesters of a degree, his skill portfolio and competence level for each skill is expected to grow. Effectively modeling these curriculum skills, mapping them to assessment tasks across subjects of a degree, and measuring the progression in learner competence level is, largely, still an unsolved problem. Previous work at this scale is limited. This systematic tracking of skills and competence is crucial for effective quality control and optimization of degree structures. Our main contribution is an architecture for a curriculum information management system to facilitate this systematic tracking of skill and competence level progression in a Computer