Andrea Britto Mattos

Exploring the Use of Massive Open Online Courses for Teaching Students with Intellectual Disability

In this paper, we report on a qualitative study that investigates the impact of using a popular Massive Open Online Course (MOOC) to complement the vocational training of students with intellectual disability (ID). We have been investigating this problem for several months in partnership with a Brazilian NGO (Non-Governmental Organization) for people with ID. Our methodology integrates different aspects of human-computer interaction (i.e., requirement gathering sessions and observation of real subjects). Potential users were involved since the beginning of this research, starting with focus groups and interviews with experts, followed by the observation of a traditional vocational training session, and then the assessment of a popular MOOC in the classroom. In this paper, we discuss the process and present our preliminary results, providing some indications on how MOOCs could better support instructors and students with ID.

Marker-assisted recognition of dynamic content in public spaces

In this work we present an image processing-based assistant for helping visually impaired citizens with the task of recognizing dynamic content within fixed layouts of displays in public spaces. Our solution relies on the placement of markers, in order to facilitate the location and recognition of target objects and, at the same time, provide hints to users about how to better position their mobile devices cameras to capture the whole information contained in the display.

Multi-view Mouth Renderization for Assisting Lip-reading

Previous work demonstrated that people who rely on lip-reading often prefer a frontal view of their interlocutor, but sometimes a profile view may display certain lip gestures more noticeably. This work refers to an assistive tool that receives an unconstrained video of a speaker, captured at an arbitrary view, and not only locates the mouth region but also displays augmented versions of the lips in the frontal and profile views. This is made using deep Generative Adversarial Networks (GANs) trained on several pairs of images. In the training set, each pair contains a mouth picture taken at a random angle and the corresponding picture (i.e., relative to the same mouth shape, person, and lighting condition) taken at a fixed view. In the test phase, the networks are able to receive an unseen mouth image taken at an arbitrary angle and map it to the fixed views -- frontal and profile. Because building a large-scale pairwise dataset is time consuming, we use realistic synthetic 3D models for training, and videos of real subjects as input for testing. Our approach is speaker-independent, language-independent, and our results demonstrate that the GAN can produce visually compelling results that may assist people with hearing impairment.

A platform to support personalized training of people with disabilities

Digital education has potential to provide different possibilities for personalization and consequently reach a larger and more diverse number of people. Personalization is a key component of solutions addressing important and long-standing pedagogical challenges in education, such as dealing with heterogeneity of learning styles. In particular scenarios where accessibility support is required, personalization depends on the creation of different representations for individual pieces of content. In this light, the main goal of this article is to describe how we addressed the challenges involved in the construction of a platform that satisfies this requirement. We thus present a system that supports the creation, adaptation, and delivery of personalized courses for people with multiple types of disabilities. More specifically, we introduce the technology, describe its main capabilities, and discuss the results of early evaluations by two instructors of an institution that provides vocational training for people with intellectual disabilities. Our initial results show that the tool was favorably assessed by the instructors and can potentially be adopted in this community.

Marker-based image recognition of dynamic content for the visually impaired

The access to information displayed in public spaces is a challenge faced by visually impaired people for which image processing techniques have the potential to deliver satisfactory solutions. However, object recognition algorithms must initially locate possible candidates in the images, which is a hard task in complex scenes. In this article, we introduce an image processing technique that relies on the incorporation of markers to panels and boards with fixed layouts displaying dynamic content. The markers allow: a) locating the objects to be recognized; b) correcting perspective in the input images; c) limiting the training set size for supervised learning; and d) guiding the visually impaired by indicating how they should position their devices for adequate pictures. The proposed technique can be used for automatic recognition of texts and images and is suitable for deployment on mobile devices, providing more independence to the citizens. Results of preliminary tests on vending machines show that this method is robust enough to be used in practice.

A Comparative Study of Technology-Mediated Learning Strategies for Teaching People with Intellectual Disability

In this paper, we report on our experiences investigating the role of digital technology in face-to-face instruction of students with intellectual disability. In this process, we used a multi-method approach and our findings integrate results from focus groups, interviews, observations, iterative prototyping, and user evaluation. Ultimately, we hope that our work can motivate future research efforts and bring to light opportunities to be considered in the development of mobile-based education solutions.

Identifying challenges and opportunities in computer-based vocational training for low-income communities of people with intellectual disabilities

Vocational training of people with disabilities (PwD) can potentially improve social and economic prospects, but at the same time, it can be significantly challenging due to the need for specialized training and technology. Unfortunately, in developing countries this problem is magnified because, in general, low-income groups have limited access to appropriate content and assistive technologies. In this paper, we present initial findings from a qualitative field study of computer-mediated vocational training for low-income students with intellectual disability (ID) in Brazils largest urban area. The observations took place during 3 computer-based training sessions, which involved 23 students with ID. Based on data gathered from observations and semi-structured interviews with 2 instructors, we describe and discuss strategies they employ to teach heterogeneous groups of students with ID. We also examine some obstacles that our participants usually face during the classes at the computer lab and present suggestions for future initiatives that could better support instructors and students with ID in the learning process.

Flower Classification for a Citizen Science Mobile App

This work describes an efficient approach for flower classification that is suitable for deployment in mobile devices, allowing its use in a citizen science application for biodiversity monitoring. In the proposed system, geo-located images are uploaded by the user and segmented semi-automatically. We propose a classification method based on histogram comparison of color, shape and texture cues, using metric learning for feature weighting. Our method is tested on the Oxford Flower Dataset and we are able to achieve state-of-the-art accuracy, while proposing an approach that can run efficiently in mobile devices.