Automatic Object Recognition in a Light-Weight Augmented Reality-based Vocabulary Learning Application for Children with Autism

Abstract

A number of previous controlled studies have underscored the importance of early diagnosis and intervention in autism. However, despite the technological advances, augmented reality-based (AR) intervention for Chinese autistic children is still rare, which motivates our study. In particular, in this paper, we present a mobile vocabulary-learning application for Chinese autistic children by creating authentic opportunities in outdoor and home use. The core object recognition module is implemented in the deep learning platform, TensorFlow, on one hundred training models; unlike other sophisticated systems, the algorithm has to run in an offline fashion. A pilot study aiming at investigating the system s feasibility and usability had been conducted with typically developing children and their parents with very promising and satisfying results. We also further tested performance of the offline learning algorithm using seven animal toys with very satisfying results. Since the current literature of AR-technology on Chinese word-learning for children with special needs is still in its infancy and arguably lacks rigor in especially design and assessment, which thus offers limited insights into its therapeutic efficacy, feasibility and applicability of individualized intervention for autistic individuals, particularly children. It is our hope that this preliminary study adds to our understanding towards the usability and usefulness of such AR-based mobile learning application.