Varied Expression Analysis of Children With ASD Using Multimodal Deep Learning Technique

Abstract

Abstract Autism spectrum disorder (ASD) is one of the prominent and complex neurodevelopment disorders that are rising around the globe. In India, the autism rate has grown and now about 1 out of 88 children is prone to autism. ASD is associated with various problems such as low vision clarity, communication difficulties, repetitive behavior, insecure emotions, sensory issues, uninterested behavior, etc. Many researchers work towards the identification of major causes of autism and various impacts of the disorder. Although the basic causes and mechanism behind ASD could not be completely identified, it is obvious that the abnormal neurotransmission in brain regions prevents the child from normal behavior and motor actions and leads to an altered behavior. This impairment of mental difficulties could be identified with regular periodic analysis of the child communication behavior, social interaction, object identification, emotional sequence, linguistic ability and knowledge adaptation rate between the ages of 10 months to 3 years. The major pitfall in the ASD is that people from rural areas are not aware of the factors, causes and behavioral changes in the child with ASD. This leads to poor and misleading training and guiding the children for their future. Also the clinical analysis involves numerous trials to analyze the autistic characteristics across children age. Such clinical analysis could be made faster with the computer vision, thus resulting in early identification of autism. This chapter mainly focuses on the emotional sequence identification of children who are ASD positive and ASD negative (i.e., normal TD). The first screening is made through the facial expression analysis in children identified as ASD positive and TD. The methodology involved in facial detection and feature analysis is Viola–Jones algorithm and facial landmark vectorization, respectively. The facial expression is then classified using an SVM linear classifier. The facial expressions of ASD and TD are compared to identify the variations in facial expressions of children. This level of screening through expression identification is made in a contactless environment as children may get influenced by the object they face. The second level of improvement in screening involves a deep learning technique to identify the children facial expressions and emotion using facial landmark analysis. The former involves plotting up of 68 points on the face identified though landmark detection. In the later methodology, computational neural network technique is employed to identify the correlation between the points for facial emotion analysis. This improves the performance of analysis in emotion identification in children in the same contactless environment. The ultimate aim of the paper is to design a complete combinatorial model for the results from various screening experiments involving a multimodal deep learning technique that projects to a better solution of autism identification. The multimodal technique is planned to take input from the screenings 1 and 2 simultaneously as initial neurons, x, and form a hidden layer, h, by selecting the appropriate neuron in fine-tuning the cross-neuron activation function. The experimental analysis identifies (i) the expression showed by every autistic child, (ii) the major expression observed among the children with ASD, (iii) expression change in a child and the pattern of expression, and (iv) expression analysis through SVM and CNN learning methods. The experimental analysis boosts in training the autistic child on a proper streamline, which helps the parent and trainer in bringing up the child to reach heights.