Olivier Augereau

A proposal of a document image reading-life log based on document image retrieval and eyetracking

Instead of analyzing directly the document images, analyzing the document reading can offer new perspectives for extracting information about both the reader and the document. Analyzing how people read texts can help to understand the cognitive process of the reading and might lead to new approaches and new solutions for pattern recognition and document image analysis. It can also lead to create smart documents that can measure reading information, provide feedback and adapt themselves depending on the behavior of the readers. As a step towards document reading analysis, the authors propose in this paper a solution for extracting the reading information and creating a “reading-life log”. This reading-life log contains basic features that can be used for many different kinds of applications. A tag cloud evolving according to the reading is presented as a first application of the reading-life log.

The augmented narrative: toward estimating reader engagement

We present the concept of bio-feedback driven computing to design a responsive narrative, which acts according to the readers experience. We explore on how to detect engagement and give our evaluation on the usefulness of different sensor modalities. We find temperature and blink frequency are best to estimate engagement and can classify engaging and non-engaging user-independent without error for a small user sample size (5 users).

Towards an automated estimation of English skill via TOEIC score based on reading analysis

Estimating automatically the degree of language skill by analyzing the eye movements is a promising way to help people from all over the world to learn a new language. In this study, we focus on the English skills of non-native speakers. Our aim is to provide an algorithm that can assess accurately and automatically the TOEIC score after reading English texts for few minutes. As a first step towards this direction, we propose an algorithm that can predict accurately this score after reading and answering some questions about the comprehension of few English texts. We use an eye tracker in order to record the eye gaze, i.e. the positions where the reader is looking at. Then we extract several features to characterize the behavior, and consequently the skill of the reader. We also add a feature based on the number of correct answers to the questions. By using a machine learning based on multivariate regression, the score is estimated user independently. A backward stepwise feature selection is used to select the relevant features and to optimize the estimation. As a main result, the TOEIC score is estimated with 21.7 points of mean absolute error for 21 subjects after reading and answering the questions of only 3 documents.

Estimation of english skill with a mobile eye tracker

Learning a foreign language such as English is an important task for many people. The process of learning takes time and it is important to have a simple way to evaluate the progress of the skill. We propose a method to evaluate the readers English skill based on a mobile eye tracking system. The eye tracker captures the readers behavior while reading a document. The front camera of the eye tracker records the scene image that contains the read document. By using a retrieval algorithm we can recognize the read document and project the eye gaze data from the scene image to the document space. Then, some features related to the reading and solving behavior on several documents are computed. As a first result, we show that the TOEIC score can be estimated with an error of 36.3 points.

Vertical error correction of eye trackers in nonrestrictive reading condition

The eye tracking technology is used for four decades for studying reading behavior. The applications are various: estimating the reader comprehension, identifying the reader, summarizing a read document, creating a reading-life log, etc. The gaze data used in such applications has to be accurate enough to perform the analysis. In order to improve the accuracy, most of the experiments are set up with restrictive conditions such as using a head fixation and a professional eye tracker. It implies that the results are valid only in restrictive laboratory settings and an unrealistic small error is produced by the experiment. However, the use of affordable eye trackers in realistic conditions of reading leads to large errors in the recordings. We propose a new algorithm to correct the vertical error and to align the gazes with the text. The proposed algorithm is robust to rereading and skipping some parts of text, contrary to all the other algorithms of the state of the art. We show that up to 69 % of the gazes are aligned with the correct text lines.

Comic visualization on smartphones based on eye tracking

The visualization of comic images on a small screen is a difficult problem as the image is too large to be displayed on the screen and we do not know which areas and in which order the users want to see the image. The basic solution for the user is to look at the image in full screen without being able to see the details, or to zoom and scroll through the image, which can be quite inconvenient if the interactions have to often be repeated. Our idea is to use an eye tracker to record where the users reading a comic on paper books or large screens are looking at, to reproduce their reading behaviors with a comic visualization system and guide the users using a smaller screen through the comic.

Reading similarity measure based on comparison of fixation sequences

The eye movement is an important source of information for the reading analysis. We propose a method for computing a similarity measure between two fixation sequences. In order to estimate the effectiveness of the similarity measure, we investigate whether a high similarity is obtained when two subjects read the same document. A F1score of 0.92 is obtained for retrieving the same document based on the reading similarity.

Eye gaze and text line matching for reading analysis

Eye tracking data has been widely used to analyze our reading behavior. Usually, experiments are carried out with head fixations or by analyzing eye tracking data in large areas such as paragraphs. But if we want to analyze the eye gaze line by line or word by word with a non invasive apparatus, we have to face the mislocation of the recorded eye gaze. The lack of accuracy involves a difficult analysis of the small eyes movements during reading. This paper proposes a method to match lines of gazes with corresponding text lines, using three different methods. We will show that the Dynamic Time Warping is a promising way to measure similarity between a line of gaze and a text line.

Assessing Cognitive Workload on Printed and Electronic Media using Eye-Tracker and EDA Wristband

With the expansion of e-learning platforms, we receive a great opportunity to learn and study just using an electronic device. In this paper, we measured the differences in information processing on screen and paper with 18 participants using an eye-tracker and an EDA wristband. Our findings show that the media type has a significant influence on cognitive workload and understandability of the content. The results of this work are of vital importance for the design of new intelligent user interfaces and reveal the necessity to take mental processes of users more into account.

A Survey of Comics Research in Computer Science

Graphical novels such as comics and mangas are well known all over the world. The digital transition started to change the way people are reading comics, more and more on smartphones and tablets and less and less on paper. In the recent years, a wide variety of research about comics has been proposed and might change the way comics are created, distributed and read in future years. Early work focuses on low level document image analysis: indeed comic books are complex, they contains text, drawings, balloon, panels, onomatopoeia, etc. Different fields of computer science covered research about user interaction and content generation such as multimedia, artificial intelligence, human-computer interaction, etc. with different sets of values. We propose in this paper to review the previous research about comics in computer science, to state what have been done and to give some insights about the main outlooks.