A. S. C. S. Sastry

Visual-verbal machine interpreter for sign language recognition under versatile video backgrounds

This research paper is an attempt to create a video background independent sign language recognition (SLR) system. SLR acts as a Machine Interpreter (MI) between a mute person and normal person. One of the key difficulties in sign language recognition is video background of a sign video in which signer is located. Signer is extracted from cluttered back video backgrounds using boundary and prior shape information. Active contours energy function is formulated by amalgamating energy functions from boundary and shape prior elements. Energy minimization for movement of active contour is achieved using Euler- Lagrange equations. Feature vector is constructed from the segmented signer frames using a frame average based pooling function along with the shape inform obtained from active contour. Artificial Neural Network is constructed to classify and recognize gestures from video frames of signers. Compared to traditional methods of sign language recognition, the proposed Visual-Verbal Machine Interpreter (V2MI) for sign language recognition offers a recognition rate of around 93%.

Block based thresholding in wavelet domain for denoising ultrasound medical images

Medical ultrasound imaging has transformed the disease identification in the human body in the last few decades. The major setback for ultrasound medical images is speckle noise. Speckle noise is created in ultrasound images due to numerous reflections of ultrasound signals from hard tissues of human body. Speckle noise corrupts the medical ultrasound images dropping the detectable quality of the image. An endeavor is made to recover the image quality of ultrasound medical images by using block based hard and soft thresholding of wavelet coefficients. Medical ultrasound image is transformed to wavelet domain using debauchees mother wavelet. Divide the approximate and detailed coefficients into uniform blocks of size 8×8, 16×16, 32×32 and 64×64. Hard and soft thresholding on these blocks of approximate and detailed coefficients are applied. Inverse transformation to original spatial domain produces a noise reduced ultrasound image. Experiments were conducted on medical ultrasound images obtained from diagnostic centers in Vijayawada, India. Quality of improved images in measured using peak signal to noise ratio (PSNR), image quality index (IQI), structural similarity index (SSEVI).

Optical Flow Hand Tracking and Active Contour Hand Shape Features for Continuous Sign Language Recognition with Artificial Neural Networks

To extract hand tracks and hand shape features from continuous sign language videos for gesture classification using backpropagation neural network. Horn Schunck optical flow (HSOF) extracts tracking features and Active Contours (AC) extract shape features. A feature matrix characterizes the signs in continuous sign videos. A neural network object with backpropagation training algorithm classifies the signs into various words sequences in digital format. Digital word sequences are translated into text with matching and the suiting text is voice translated using windows application programmable interface (Win-API). Ten signers, each doing sentences having 30 words long tests the performance of the algorithm by computing word matching score (WMS). The WMS is varying between 88 and 91 percent when executed on different cross platforms on various processors such as Windows8 with Inteli3, Windows8.1 with inteli3 and windows10 with inteli3 running MATLAB13(a).

Crowd Density Analysis and tracking

Crowd Density Analysis (CDA) aims to compute concentration of crowd in surveillance videos. This paper core is to estimate the crowd concentrations using crowd feature tracking with optical flow. Local features are extracted using Features for Accelerated Segment Test (FAST) algorithm per frame. Optical flow tracks the features between frames of the surveillance video. This process identifies the crowd features in consecutive frames. Kernel density estimator computes the crowed density in each successive frame. Finally individual people are tracked using estimated flows. The drawback of this method is similar to suffered by most of the estimation methods in this class that is reliability. Hence testing with three popular optical flow models is initiated to find the best optical flow. Three methods are Horn-Schunck (HSOF), Lukas-Kanade (LKOF) and Correlation optical flow (COF). Five features extraction methods were tested along with the three optical flow methods. FAST features with horn-schunck estimates crowed density better than the remaining methods. People tracking application with this algorithm gives good tracks compared to other methods.

Indian Classical Dance Action Identification and Classification with Convolutional Neural Networks

Extracting and recognizing complex human movements from unconstrained online/offline video sequence is a challenging task in computer vision. This paper proposes the classification of Indian classical dance actions using a powerful artificial intelligence tool: convolutional neural networks (CNN). In this work, human action recognition on Indian classical dance videos is performed on recordings from both offline (controlled recording) and online (live performances, YouTube) data. The offline data is created with ten different subjects performing 200 familiar dance mudras/poses from different Indian classical dance forms under various background environments. The online dance data is collected from YouTube for ten different subjects. Each dance pose is occupied for 60 frames or images in a video in both the cases. CNN training is performed with 8 different sample sizes, each consisting of multiple sets of subjects. The remaining 2 samples are used for testing the trained CNN. Different CNN architectures were designed and tested with our data to obtain a better accuracy in recognition. We achieved a 93.33% recognition rate compared to other classifier models reported on the same dataset.

Selfie Continuous Sign Language Recognition with Neural Network Classifier

This works objective is to bring sign language closer to real-time implementation on mobile platforms with a video database of Indian sign language created with a mobile front camera in selfie mode. Pre-filtering, segmentation, and feature extraction on video frames creates a sign language feature space. Artificial Neural Network classifier on the sign feature space are trained with feed forward nets and tested. ASUS smart phone with 5M pixel front camera captures continuous sign videos containing an average of 220 frames for 18 single-handed signs at a frame rate of 30 fps. Sobel edge operator’s power is enhanced with morphology and adaptive thresholding giving a near perfect segmentation of hand and head portions. Word matching score (WMS) gives the performance of the proposed method with an average WMS of around 90% for ANN with an execution time of 0.5221 s during classification. Fully novel method of implementing sign language to introduce sign language recognition systems on smart phones for making it a real-time usage application.

Sign Language Conversion Tool (SLCTooL) Between 30 World Sign Languages

This paper proposes to find similarity between sign language finger spellings of alphabets from 30 countries with computer vision and support vector machine classifier. A database of 30 countries sign language alphabets is created in laboratory conditions with nine test subjects per country. Binarization of sign images and subsequent feature extraction with histogram of oriented gradients gives a feature vector. Classification with support vector machine provides insight into the similarity between world sign languages. The results show a similarity of 61% between Indian sign language and Bangladesh sign language belonging to the same continent, whereas the similarity is 11 and 7% with American and French sign languages in different continents. The overall classification rate of multiclass support vector machine is 95% with histogram of oriented gradient features when compared to other feature types. Cross-validation of the classifier is performed by finding an image structural similarity measure with Structural Similarity Index Measure.

3D Motion Capture for Indian Sign Language Recognition (SLR)

A 3D motion capture system is being used to develop a complete 3D sign language recognition (SLR) system. This paper introduces motion capture technology and its capacity to capture human hands in 3D space. A hand template is designed with marker positions to capture different characteristics of Indian sign language. The captured 3D models of hands form a dataset for Indian sign language. We show the superiority of 3D hand motion capture over 2D video capture for sign language recognition. 3D model dataset is immune to lighting variations, motion blur, color changes, self-occlusions and external occlusions. We conclude that 3D model based sign language recognizer will provide full recognition and has a potential for development of a complete sign language recognizer.

Indian sign language recognition using graph matching on 3D motion captured signs

A machine cannot easily understand and interpret three-dimensional (3D) data. In this study, we propose the use of graph matching (GM) to enable 3D motion capture for Indian sign language recognition. The sign classification and recognition problem for interpreting 3D motion signs is considered an adaptive GM (AGM) problem. However, the current models for solving an AGM problem have two major drawbacks. First, spatial matching can be performed on a fixed set of frames with a fixed number of nodes. Second, temporal matching divides the entire 3D dataset into a fixed number of pyramids. The proposed approach solves these problems by employing interframe GM for performing spatial matching and employing multiple intraframe GM for performing temporal matching. To test the proposed model, a 3D sign language dataset is created that involves 200 continuous sentences in the sign language through a motion capture setup with eight cameras.The method is also validated on 3D motion capture benchmark action dataset HDM05 and CMU. We demonstrated that our approach increases the accuracy of recognizing signs in continuous sentences.

SWIFT cognitive behavioral assessment model built on cognitive analytics of empirical mode internet of things

This paper introduces a study and analysis to predict the present human behaviour through his/her object interactions in the physical environment. The physical environment consists of a door, chair and telephone with accelerometer sensors attached to them and connected to computer using a raspberry pi IoT (Internet of Things) kit. Two other parameters used for assessment are human voice intensities and human motion analysis through a motion capture camera with inbuilt microphone and Wi-Fi module. The dataset is a collection of accelerometer data from chair and telephone, human interaction with door through camera and voice sample of a word ‘Hello’. These 4 parameter measurements are collected from 15 test subjects in the age group 19-21 without their knowledge. We used the dataset to train and test 3 predominant behaviours in the chosen age group namely, excitable, assertive and pleasant on an artificial neural network with backpropagation training algorithm. The overall recognition accuracy is 84.89% based on the physical assessment from a physiatrist of all the test subjects. This study can help individuals, doctors and machines to predict the current human emotional state and provide feedback to modify unpleasant current state of behaviour to a pleasant state to maximize human performance.