Zdenek Zdrahal

Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment

One of the key interests for learning analytics is how it can be used to improve retention. This paper focuses on work conducted at the Open University (OU) into predicting students who are at risk of failing their module. The Open University is one of the worlds largest distance learning institutions. Since tutors do not interact face to face with students, it can be difficult for tutors to identify and respond to students who are struggling in time to try to resolve the difficulty. Predictive models have been developed and tested using historic Virtual Learning Environment (VLE) activity data combined with other data sources, for three OU modules. This has revealed that it is possible to predict student failure by looking for changes in users activity in the VLE, when compared against their own previous behaviour, or that of students who can be categorised as having similar learning behaviour. More focused analysis of these modules applying the GUHA (General Unary Hypothesis Automaton) method of data analysis has also yielded some early promising results for creating accurate hypothesis about students who fail.

Using Ontologies for Defining Tasks, Problem-Solving Methods and their Mappings

In recent years two main technologies for knowledge sharing and reuse have emerged: ontologies and problem solving methods (PSMs). Ontologies specify reusable conceptualizations which can be shared by multiple reasoning components communicating during a problem solving process. PSMs describe in a domain-independent way the generic reasoning steps and knowledge types needed to perform a task. Typically PSMs are specified in a task-specific fashion, using modelling frameworks which describe their control and inference structures as well as their knowledge requirements and competence. In this paper we discuss a novel approach to PSM specification, which is based on the use of formal ontologies. In particular our specifications abstract from control, data flow and other dynamic aspects of PSMs to focus on the logical theory associated with a PSM (method ontology). This approach concentrates on the competence and knowledge requirements of a PSM, rather than internal control details, thus enabling black-box-style reuse. In the paper we also look at the nature of PSM specifications and we show that these can be characterised in a task-independent style as generic search strategies. The resulting ‘modelling gap’ between method-independent task specifications and task-independent method ontologies can be bridged by constructing the relevant adapter ontology, which reformulates the method ontology in task-specific terms. An important aspect of the ontology-centred approach described here is that, in contrast with other characterisations of task-independent PSMs, it does away with the simple, binary distinction between weak and strong methods. We argue that any method can be defined in either task-independent or task-dependent style and therefore such distinction is of limited utility in PSM reuse. The differences between PSMs which affect reuse concern the ontological commitments which they make with respect to domain knowledge and goal specifications.

Story fountain: intelligent support for story research and exploration

Increasingly heritage institutions are making digital artifacts available to the general public and research groups to promote the active exploration of heritage and encourage visits to heritage sites. Stories, such as folklore and first person accounts form a useful and engaging heritage resource for this purpose. Story Fountain provides intelligent support for the exploration of digital stories. The suite of functions provided in Story Fountain together support the investigation of questions and topics that require the accumulation, association or induction of information across the story archive. Story Fountain provides specific support toward this end such as for comparing and contrasting story concepts, the presentation of story paths between concepts, and mapping stories and events according to properties such as who met whom and who lived where.

A generic library of problem solving methods for scheduling applications

In this paper, we propose a generic library of problem-solving methods for scheduling applications. Although some attempts have been made in the past at developing the libraries of scheduling problem-solvers, these only provide limited coverage. Many lack generality, as they subscribe to a particular scheduling domain. Others simply implement a particular problem-solving technique, which may be applicable only to a subset of the space of scheduling problems. In addition, most of these libraries fail to provide the required degree of depth and precision. In our approach, we subscribe to the task-method-domain-application knowledge modeling framework which provides a structured organization for the different components of the library. At the task level, we construct a generic scheduling task ontology to formalize the space of scheduling problems. At the method level, we construct a generic problem-solving model of scheduling that generalizes from the variety of approaches to scheduling problem-solving, which can be found in the literature. The generic nature of this model is demonstrated by constructing seven methods for scheduling as an alternative specialization of the model. Finally, we validated our library on a number of applications to demonstrate its generic nature and effective support for developing scheduling applications

Solving VT in VITAL: a study in model construction and knowledge reuse

In this paper we discuss a solution to the Sisyphus II elevator design problem developed using the VITAL approach to structured knowledge-based system development. In particular we illustrate in detail the process by which an initial model of Propose & Revise problem solving was constructed using a generative grammar of model fragments and then refined and operationalized in the VITAL operational conceptual modelling language (OCML). In the paper we also discuss in detail the properties of a particular Propose & Revise architecture, called “Complete-Model-then-Revise”, and we show that it compares favourably in terms of competence with alternative Propose & Revise models. Moreover, using as an example the VT domain ontology provided as part of the Sisyphus II task, we critically examine the issues affecting the development of reusable ontologies. Finally, we discuss the performance of our problem solver and we show how we can use machine learning techniques to uncover additional strategic knowledge not present in the VT domain.

A methodological approach to supporting organizational learning

Many organizations need to respond quickly to change and their workers need to regularly develop new knowledge and skills. The prevailing approach to meeting these demands is on-the-job training, but this is known to be highly ineffective, cause stress and devalue workplace autonomy. Conversely, organizational learning is a process through which workers learn gradually in the work context through experience, reflection on work practice and collaboration with colleagues. Our approach aims to support and enhance organizational learning around enriched work representations. Work representations are tools and documents used to support collaborative working and learning. These are enriched through associations with formal knowledge models and informal discourse. The work representations, informal discourse and associated knowledge models together form on organizational memory from which knowledge can be retrieved later. Our methodological approach to supporting organizational learning is drawn from three industrial case studies concerned with machine maintenance, team planning and hotline support. The methodology encompasses development and design activities, a description of the roles and duties required to sustain the long-term use of the tools, and applicability criteria outlining the kind of organizations that can benefit from this approach.

A library of problem-solving components based on the integration of the search paradigm with task and method ontologies

In this paper we investigate the reuse of tasks and problem-solving methods and we propose a model of how to organize a library of reusable components for knowledge-based systems. In our approach, we first describe a class of problems by means of atask ontology. Then we instantiate a generic model of problem solving assearchin terms of the concepts in the task ontology, to derive a task-specific, but method-independent,problem-solving model. Individual problem-solving methods can then be (re-)constructed from the generic problem-solving model through a process ofontology/method specializationandconfiguration. The resulting library of reusable components enjoys a clear theoretical basis and has been tested successfully on a number of applications. In the paper, we illustrate the approach in the area ofparametric design.

Worlds and transformations: Supporting the sharing and reuse of engineering design knowledge

Design involves the formulation of a solution, such as a product specification, from initial requirements. Design in industrial and other contexts often involves the building and use of models that allow the designer to test hypotheses and learn from possible design decisions prior to building the physical product. The building and testing of models is a design process in its own right. Previous work in knowledge management, design rationale and the psychology of design has demonstrated that designers often vary from prescriptive methodologies of the design process and have problems appropriately describing their design activity in order to support design collaboration and the reuse of design artefacts. Drawing on this work, we support design collaboration and reuse structured according to key transformational episodes in the design process and the design artefacts they produce. To support this, we characterise the design task as progressing through a series of worlds, each comprising its own concepts and vocabulary, and supported by its own design tools. The design process can then be described in terms of important transformations that are made from one world to the next. This allows a targeted approach to rationale capture integrated with work practice and associated with products of the design process. This approach has been successfully deployed and tested in two industrial engineering companies. Findings included improved collaboration in design teams, effective reuse and improved training for new members of the design team. This work has more general implications for the development of design rationale methods and tools to support the design process.

Predicting Student Performance from Combined Data Sources

This chapter will explore the use of predictive modeling methods for identifying students who will benefit most from tutor interventions. This is a growing area of research and is especially useful in distance learning where tutors and students do not meet face to face. The methods discussed will include decision-tree classification, support vector machine (SVM), general unary hypotheses automaton (GUHA), Bayesian networks, and linear and logistic regression. These methods have been trialed through building and testing predictive models using data from several Open University (OU) modules. The Open University offers a good test-bed for this work, as it is one of the largest distance learning institutions in Europe. The chapter will discuss how the predictive capacity of the different sources of data changes as the course progresses. It will also highlight the importance of understanding how a student’s pattern of behavior changes during the course.

Implementing a Learning Analytics Intervention and Evaluation Framework: What Works?

Substantial progress in learning analytics research has been made in recent years to predict which groups of learners are at risk. In this chapter, we argue that the largest challenge for learning analytics research and practice still lies ahead of us: using learning analytics modelling, which types of interventions have a positive impact on learners’ Attitudes, Behaviour and Cognition (ABC). Two embedded case-studies in social science and science are discussed, whereby notions of evidence-based research are illustrated by scenarios (quasi-experimental, A/B-testing, RCT) to evaluate the impact of interventions. Finally, we discuss how a Learning Analytics Intervention and Evaluation Framework (LA-IEF) is currently being implemented at the Open University UK using principles of design-based research and evidence-based research.