Leana Copeland

Predicting reading comprehension scores from eye movements using artificial neural networks and fuzzy output error

Predicting reading comprehension from eye gaze data is a difficult task. We investigate the use of artificial neural networks(ANNs) to predict reading comprehension scores from eye gaze collected from participants who read and completed an onlinetutorial in our lab. Problems such as large feature sets and small highly imbalanced data sets compound to make this task evenmore complex. We propose using fuzzy output error (FOE) as an alternative performance function to mean square error (MSE)for training feed-forward neural networks to overcome these problems. We show that the use of FOE as the performance functionfor training ANNs provides significantly better classification of eye movements to reading comprehension scores. ANNs withthree hidden layers of neurons gave the best classification results especially when FOE is used as the performance functionfor training. In these cases we found up to 50% reduction in misclassification rates compared to using MSE. We found thatANNs give optimal classification results in comparison to other classification techniques. When FOE is used as the performancefunction for training the ANNs the misclassification rates are halved compared to the other techniques. Cluster analysis wasperformed on one of the more complex data sets. Interesting reading behaviour properties were found within the data set.The intended use of this research is in the design of adaptive online learning environments that use eye gaze to predict usercomprehension from reading behavior.

The effect of subject familiarity on comprehension and eye movements during reading

We investigate factors affecting reading and overall comprehension of the underlying meaning and concepts within a piece of text using eye movements. Our objective is to identify eye movement measures that will predict reading comprehension, and intend to apply them in eLearning to create dynamic learning environments that can use eye movement to detect reader comprehension. We found that the self-reported familiarity of readers with the subject of documents affects their reading behaviour but not their total comprehension score, and found that we could identify answer-seeking behaviour and a measure of their actual familiarity with the text content using eye gaze.

Fuzzy Output Error as the Performance Function for Training Artificial Neural Networks to Predict Reading Comprehension from Eye Gaze

Imbalanced data sets are common in real life and can have a negative effect on classifier performance. We propose using fuzzy output error (FOE) as an alternative performance function to mean square error (MSE) for training feed forward neural networks to overcome this problem. The imbalanced data sets we use are eye gaze data recorded from reading and answering a tutorial and quiz. The goal is to predict the quiz scores for each tutorial page. We show that the use of FOE as the performance function for training neural networks provides significantly better classification of eye movements to reading comprehension scores. A neural network with three hidden layers of neurons gave the best classification results especially when FOE was used as the performance function for training. In these cases, upwards of a 19% reduction in misclassification was achieved compared to using MSE as the performance function.

Tutorials in eLearning—How Presentation Affects Outcomes

The presentation of learning materials affects how we learn. In this paper, we use eye tracking to investigate how different sequences of text and test questions affect performance outcomes, eye movements, and reading behavior for first (L1) English language and second (L2) English language readers. We show that different presentation sequences induce different performance outcomes, eye movements, and reading behavior. The sequence can affect how a participant reads the text as well as their perceptions of how well they understood what they read. For instance, if questions and text are not shown together, this improves participants’ ability to accurately perceive their comprehension and promotes thorough reading. Alternatively, showing questions before the text promotes skimming behavior. Importantly, the presentation sequence affects both L1 and L2 readers in the same way. We observe that L2 reader take longer to read text but have the same comprehension levels as L1 readers; this difference comes primarily from longer fixation durations. The results from this paper can be used to design learning materials in eLearning environments to influence how students interact with the learning environment as well as how they learn. The purpose of this investigation is to make informative decisions about designing adaptive eLearning environments.

Visual Distractions Effects on Reading in Digital Environments: A Comparison of First and Second English Language Readers

Reading in digital environments can be very distracting. Using eye-tracking technology, we investigate if text readability affects distraction rate, eye movements, and reading comprehension in a visually distracting digital environment. We compared an easy-to-read text and a hard-to-read text on both first language English (L1) readers and second language English (L2) readers. Text readability was measured using the standard readability formulas such as the Flesch-Kincaid Grade level. Results show that text readability does cause different eye movements and produce reading comprehension results that deviate from what is normally expected. Readers are affected more by the distractions when reading easy-to-read text compared to when reading hard-to-read text. Furthermore, L2 readers are affected more than L1 readers. These findings can be used in the design of eLearning materials when distractions cannot be overcome.

What are You Reading Most: Attention in eLearning

Abstract Eye tracking is useful for investigating how people read and the attention that they give to certain items. We investigated how much participants read parts of educational text when they are required to answer questions relating to it. We found that there is no difference between the normalized number of fixations observed when participants answered multiple-choice questions correctly compared to when they answered incorrectly, however, there are differences for fill-in-the-blanks questions. Different presentation formats of the text and questions have an effect on the how thoroughly paragraphs containing answers to questions are read. For formats where only text is presented the first or last paragraphs are read the most thoroughly. For formats where the questions are shown with text, the fill-in-the-blanks questions were read more thoroughly than other parts on the page. This can be used to influence how students learn material in eLearning environments.

Framework for Dynamic Text Presentation in eLearning

Abstract We present the framework of an eLearning environment that can adapt to a students reading behaviour by dynamically selecting and presenting text-based learning material. The students eye gaze would be used to predict their comprehension level and the text difficulty will be altered to reflect this. This can be used to influence how students interact with the learning environment as well as how they learn the material, streamlining the learning process and optimising learning outcomes. For this framework to be viable two aspects of the design must be feasible; the first is that reading comprehension can be predicted from the readers eye movements and the second is that changing text difficulty has an effect on learning behaviour. We present preliminary results from a study investigating the latter aspect that supports this claim.

Effect of presentation on reading behaviour

Eye tracking is a useful tool for investigating how people read and the attention that they give to certain words and phrases. Eye tracking is used to investigate how different presentation formats of the same learning material affect learning performance, eye movements, and reading behaviour. We show that different presentation formats induce different eye movements and that reading behaviour is subject to the goals placed on the reader. We also observe that the presentation format affects not only their learning performance but also how they perceive their performance. Finally, we show that different formats and question types can induce specific reading behaviour such as thorough reading. This can be used to influence how students interact with the learning environment as well as how they learn the material. The purpose of this investigation is to be able to make informative decisions about designing adaptive eLearning environments.

Fuzzy Signature Neural Networks for Classification: Optimising the Structure

We construct fuzzy signature neural networks where fuzzy signatures replace hidden neurons in a neural network similar to a radial basis function neural network. We investigated the properties of a naive and a principled approach to fuzzy signature construction. The naive approach provides very good results on benchmark datasets, but is outperformed by the principled approach when we approximate the noisy nature of real world datasets by randomly eliminating 20% of the data. The major benefit of the principled approach is to substantially improve robustness of the fuzzy signature neural networks we produce.