Dehai Zhao

Emotion Recognition from Chinese Speech for Smart Affective Services Using a Combination of SVM and DBN

Accurate emotion recognition from speech is important for applications like smart health care, smart entertainment, and other smart services. High accuracy emotion recognition from Chinese speech is challenging due to the complexities of the Chinese language. In this paper, we explore how to improve the accuracy of speech emotion recognition, including speech signal feature extraction and emotion classification methods. Five types of features are extracted from a speech sample: mel frequency cepstrum coefficient (MFCC), pitch, formant, short-term zero-crossing rate and short-term energy. By comparing statistical features with deep features extracted by a Deep Belief Network (DBN), we attempt to find the best features to identify the emotion status for speech. We propose a novel classification method that combines DBN and SVM (support vector machine) instead of using only one of them. In addition, a conjugate gradient method is applied to train DBN in order to speed up the training process. Gender-dependent experiments are conducted using an emotional speech database created by the Chinese Academy of Sciences. The results show that DBN features can reflect emotion status better than artificial features, and our new classification approach achieves an accuracy of 95.8%, which is higher than using either DBN or SVM separately. Results also show that DBN can work very well for small training databases if it is properly designed.

A Distributed Video Management Cloud Platform Using Hadoop

Due to complexities of big video data management, such as massive processing of large amount of video data to do a video summary, it is challenging to effectively and efficiently store and process these video data in a user friendly way. Based on the parallel processing and flexible storage capabilities of cloud computing, in this paper, we propose a practical massive video management platform using Hadoop, which can achieve a fast video processing (such as video summary, encoding, and decoding) using MapReduce, with good usability, performance, and availability. Red5 streaming media server is used to get video stream from Hadoop distributed file system, and Flex is used to play video in browsers. A user-friendly interface is designed for managing the whole platform in a browser-server style using J2EE. In addition, we show our experiences on how to fine-tune the Hadoop to get optimized performance for different video processing tasks. The evaluations show that the proposed platform can satisfy the requirements of massive video data management.

Food Image Recognition with Convolutional Neural Networks

In this paper, we propose a food image recognition system with convolutional neural networks(CNN), which has been applied to image recognition successfully in the literature. A CNN which consists of five layers has been built and two group of controlled trials have been processed on it. Two datasets are prepared: one is UEC-FOOD100 dataset which is an open 100-class food image dataset including about 15000 images and the other is a fruit dataset that established by ourselves including over 40000 images. We have achieved the best accuracy of 80.8% on the fruit dataset and 60.9% on the multi-food dataset. In addition, we validate the method on two groups of controlled trials and discover the effect of color under various conditions that the color feature is not always helpful for improving the accuracy by comparing the results of two group of controlled trials. As future work, we will combine image segmentation with image recognition to get a better performance.

Patching by automatically tending to hub nodes based on social trust

Malicious code can propagate rapidly via software vulnerabilities. In order to prevent the explosion of malicious codes on the Internet, a distributed patching mechanism is proposed in which the patch can tend to hub nodes automatically based on social computing in social networks. A server in social network generates automatic patches and then selects those nodes with maximum degree to push automatic patches to. Those hub nodes then send the patch to their buddies according to their degree in social network. Automatic patches propagate rapidly through hub nodes and patch nodes in social network, which will improve the security of the whole social network. Those receivers accept the patch according to trust value to the sender, which can avoid some malicious codes exploit our scheme to propagate themselves. Experiments show this mechanism is more efficient than other patching mechanisms. A distributed patching scheme which can improve the security of the InternetThe patch can tend to hub nodes automatically.A sender pushes the patch to its buddies according to their degree in social network.The receivers accept the patch according to trust value to the sender.Automatic patches propagate rapidly in social network and patch nodes.

Deep learning and SVM‐based emotion recognition from Chinese speech for smart affective services

Emotion recognition is challenging for understanding people and enhances human–computer interaction experiences, which contributes to the harmonious running of smart health care and other smart services. In this paper, several kinds of speech features such as Mel frequency cepstrum coefficient, pitch, and formant were extracted and combined in different ways to reflect the relationship between feature fusions and emotion recognition performance. In addition, we explored two methods, namely, support vector machine (SVM) and deep belief networks (DBNs), to classify six emotion status: anger, fear, joy, neutral status, sadness, and surprise. In the SVM‐based method, we used SVM multi‐classification algorithm to optimize the parameters of penalty factor and kernel function. With DBN, we adjusted different parameters to achieve the best performance when solving different emotions. Both gender‐dependent and gender‐independent experiments were conducted on the Chinese Academy of Sciences emotional speech database. The mean accuracy of SVM is 84.54%, and the mean accuracy of DBN is 94.6%. The experiments show that the DBN‐based approach has good potential for practical usage, and suitable feature fusions will further improve the performance of speech emotion recognition. Copyright

Distributed embedded deep learning based real-time video processing

There arises the needs for fast processing of continuous video data using embedded devices, for example the one needed for UAV aerial photography. In this paper, we proposed a distributed embedded platform built with NVIDIA Jetson TX1 using deep learning techniques for real time video processing, mainly for object detection. We design a Storm based distributed real-time computation platform and ran object detection algorithm based on convolutional neural networks. We have evaluated the performance of our platform by conducting real-time object detection on surveillance video. Compared with the high end GPU processing of NVIDIA TITAN X, our platform achieves the same processing speed but a much lower power consumption when doing the same work. At the same time, our platform had a good scalability and fault tolerance, which is suitable for intelligent mobile devices such as unmanned aerial vehicles or self-driving cars.

Deep Learning Based Emotion Recognition from Chinese Speech

Emotion Recognition is challenging for understanding people and enhance human computer interaction experiences. In this paper, we explore deep belief networks DBN to classify six emotion status: anger, fear, joy, neutral status, sadness and surprise using different features fusion. Several kinds of speech features such as Mel frequency cepstrum coefficient MFCC, pitch, formant, et al., were extracted and combined in different ways to reflect the relationship between feature combinations and emotion recognition performance. We adjusted different parameters in DBN to achieve the best performance when solving different emotions. Both gender dependent and gender independent experiments were conducted on the Chinese Academy of Sciences emotional speech database. The highest accuracy was 94.6i?ź%, which was achieved using multi-feature fusion. The experiment results show that DBN based approach has good potential for practical usage of emotion recognition, and suitable multi-feature fusion will improve the performance of speech emotion recognition.

An intelligent power distribution service architecture using cloud computing and deep learning techniques

Smart management of power consumption for green living is important for sustainable development. Existing approaches could not provide a complete solution for both smart monitoring of electricity consumption, and also intelligent processing of the collected data effectively. This paper presents a cloud-based intelligent power distribution service architecture, where an intelligent electricity box (IEB) is designed using Zigbee and Raspberry Pi, and a standard MQTT (Message Queuing Telemetry Transport) protocol is used to transfer monitored data to the backend Cloud computing infrastructure using open source software packages. The IEB provides cloud services of real-time electricity information checking, power consumption monitoring, and remote control of switches. The current and historical data are stored in HBase and analyzed using Long Short Term Memory (LSTM). Evaluations and practical usage show that our proposed solution is very efficient in terms of availability, performance, and the deep learning based approach has better prediction accuracy than that of both classical SVR based approach and the latest XGBoost approach.

Multi-source data fusion using deep learning for smart refrigerators

Abstract Food recognition is one of the core functions for a smart refrigerator. But there are many challenges for accurate food recognition due to reasons of too many kinds of food inside the refrigerator which tends to obscure each other, and they may look very similar. This paper proposes a fruit recognition approach that fuses weight information and multi deep learning models. The proposed approach can remarkably improve recognition accuracy. We have extensively evaluated the proposed approach for its performance and accuracy, which demonstrate the effectiveness of the proposed approach.