Lorenzo Vigentini

Visualising Virtual Learning Environments: Case Studies of the Website Exploration Tool

In this paper we describe the long term evaluation of the Website Exploration Tool (WET), an exploratory system for visualising web data, through the assessment of two Virtual Learning Environments (VLE). VLEs provide log files that can be studied with web mining techniques to understand the behaviour of the students and consequently improve the pedagogy. However, statistical metrics are not always easy to interpret for the educators, which suggests the need to develop novel approaches for the easy discovery of usage patterns. The integration of WET in the assessment process of two VLEs gave us the opportunity to conduct long-term case studies that supported the evaluation of the visual approaches used in our tool. Our main contributions are the description of the benefits of such approaches for the analysis of VLEs as well as recommended features for supporting this task, and a summary of the main findings from our case studies.

Evaluating the 'Student' Experience in MOOCs

Whilst most research on MOOCs makes inferences about the experience of learners from their interaction with the platform, few considered the rich feedback provided by learners. This paper presents the application of a conceptual model of student experience borrowed from higher education. Its relevance in the context of MOOCs was tested by using a range of questions and presentation methods in four MOOCs selected for their specific features. With varying response rates, results from over 8900 participants show how universities might view and evaluate the experience in MOOCs compared with that in traditional courses.

FutureLearn data: what we currently have, what we are learning and how it is demonstrating learning in MOOCs

Compared to other platforms such as Coursera and EdX, FutureLearn is a relatively new player in the MOOC arena and received limited coverage in the Learning Analytics and Educational Data Mining research. Founded by a partnership between the Open University in the UK, the BBC, The British Library and (originally) 12 universities in the UK, FutureLearn has two distinctive features relevant to the way their data is displayed and analyzed: 1) it was designed with a specific educational philosophy in mind which focuses on the social dimension of learning and 2) every learning activity provide opportunities for formal discussion and commenting. This workshop provides an opportunity to invite contributions and connect individual and groups to share their research activities on an international stage. As the first of its kind, this workshop will bring in a number of scholars and practitioners, as well as data scientists and analyst involved in the reporting, researching and developments emerging from the data offered by the platform.

Overcoming the MOOC Data Deluge with Learning Analytic Dashboards

With the proliferation of MOOCs and the large amount of data collected, a lot of questions have been asked about their value and effectiveness. One of the key issues emerging is the difficulty in the sense—making from the data available. The use of analytic dashboards has been suggested to provide quick insights and distil the large volume of learner interaction data generated. These dashboards hold the promise of providing a contextualized view of data and facilitating useful research exploration. However, little has been done in defining how these dashboards should be created, often resulting in a proliferation of systems for each new research agenda. We present our experience of building MOOC dashboards for two different platforms, namely Coursera and FutureLearn, motivated by a set of design goals with input from a diverse set of stakeholders. We demonstrate the features of the system and how it has served to make data accessible and useable. We report on problems faced, drawing on analyses of think-aloud sessions conducted with real educators, which have informed our dashboard process.

An Iterative Algorithm for Reputation Aggregation in Multi-dimensional and Multinomial Rating Systems

Online rating systems are widely accepted as a means for quality assessment on the web, and users increasingly rely on these systems when deciding to purchase an item online. This fact motivates people to manipulate rating systems by posting unfair rating scores for fame or profit. Therefore, both providing useful realistic rating scores as well as detecting unfair behaviours are of very high importance. Existing solutions are mostly majority based, also employing temporal analysis and clustering techniques. However, they are still vulnerable to unfair ratings. They also ignore distance between options, provenance of information and different dimensions of cast rating scores while computing aggregate rating scores and trustworthiness of raters. In this paper, we propose a robust iterative algorithm which leverages the information in the profile of raters, provenance of information and a prorating function for the distance between options to build more robust and informative rating scores for items as well as trustworthiness of raters. We have implemented and tested our rating method using both simulated data as well as three real world datasets. Our tests demonstrate that our model calculates realistic rating scores even in the presence of massive unfair ratings and outperforms well-known ranking algorithms.