Shiying Sun

Remember like humans: Visual tracking with cognitive psychological memory model

Visual tracking is a challenging computer vision task due to the significant observation changes of the target. By contrast, the tracking task is relatively easy for humans. In this article, we propose a tracker inspired by the cognitive psychological memory mechanism, which decomposes the tracking task into sensory memory register, short-term memory tracker, and long-term memory tracker like humans. The sensory memory register captures information with three-dimensional perception; the short-term memory tracker builds the highly plastic observation model via memory rehearsal; the long-term memory tracker builds the highly stable observation model via memory encoding and retrieval. With the cooperative models, the tracker can easily handle various tracking scenarios. In addition, an appearance-shape learning method is proposed to update the two-dimensional appearance model and three-dimensional shape model appropriately. Extensive experimental results on a large-scale benchmark data set demonstrate that the proposed method outperforms the state-of-the-art two-dimensional and three-dimensional trackers in terms of efficiency, accuracy, and robustness.

COMPUTER VISION-BASED DETECTION AND STATE RECOGNITION FOR DISCONNECTING SWITCH IN SUBSTATION AUTOMATION

State recognition in disconnecting switches is important during substation automation. Here, an effective computer vision-based automatic detection and state recognition method for disconnecting switches is proposed. Taking advantage of some important prior knowledge about a disconnecting switch, the method is designed using two important features of the fixed-contact facet of such disconnecting switches. First, the Histograms of Oriented Gradients (HOG) of the fixed-contact are used to design a Linear Discriminant Analysis (LDA) target detector to position the disconnecting switches and distinguish their loci against a usual cluttered background. Then a discriminative Norm Gradient Field (NGF) feature is used to train the Support Vector Machine (SVM) state classifier to discriminate disconnecting switch states. Finally, experimental results, compared with other methods, demonstrate that the proposed method is effective and achieves a low miss rate while delivering high performance in both precision and recall rate. In addition, the adopted approach is efficient and has the potential to work in practical substation automation scenarios.

Spatial-temporal context-aware abnormal event detection based on incremental sparse combination learning

Real-time abnormal event detection in practical video surveillance has been a difficult task, because there are a huge amount of continuous arrival video data, where normal events may change and only a small portion of video data contains abnormal events. In this paper, to address this problem, we use the latter arrived data to online update our model in an incremental way. We propose a spatial-temporal context-aware abnormal event detection method based on incremental sparse combination learning (ISCL). To better represent an event, we propose a novel Gradient Local Binary Patterns on Orthogonal Planes (GLBPOP) which is extracted by combining 3D gradient information and spatial-temporal context information together on two orthogonal planes. Then we propose an ISCL framework to update each dictionary in combination set in an incremental way to adapt to the possible varied upcoming normal samples. Experiments on public dataset demonstrate the proposed method is effective and superior.

A PCA–CCA network for RGB-D object recognition

Object recognition is one of the essential issues in computer vision and robotics. Recently, deep learning methods have achieved excellent performance in red-green-blue (RGB) object recognition. However, the introduction of depth information presents a new challenge: How can we exploit this RGB-D data to characterize an object more adequately? In this article, we propose a principal component analysis–canonical correlation analysis network for RGB-D object recognition. In this new method, two stages of cascaded filter layers are constructed and followed by binary hashing and block histograms. In the first layer, the network separately learns principal component analysis filters for RGB and depth. Then, in the second layer, canonical correlation analysis filters are learned jointly using the two modalities. In this way, the different characteristics of the RGB and depth modalities are considered by our network as well as the characteristics of the correlation between the two modalities. Experimental results on the most widely used RGB-D object data set show that the proposed method achieves an accuracy which is comparable to state-of-the-art methods. Moreover, our method has a simpler structure and is efficient even without graphics processing unit acceleration.

RGB-D object recognition based on RGBD-PCANet learning

In this paper, a simple deep learning method namely RGBD-PCANet is proposed for object recognition effectively. The proposed method extends the original PCANet for RGB-D images. Firstly, the RGB and depth images are preprocessed to meet the requirement of the network input layer. Secondly, features of RGB-D images are extracted by the two stages RGBD-PCANet which consists of cascaded PCA, binary hashing, and block-wise histograms. Finally, the SVM method is used as classifier. We evaluate the proposed method on the popular Washington RGB-D Object dataset. Extensive experiments demonstrate that the proposed RGBD-PCANet method achieves comparable performance to state-of-the-art CNN-based methods and the runtimes are low without GPU acceleration.

Working clothes detection of substation workers based on the image processing

On account of the substation is a basis and important element of the power system, its maintenance plays a pivotal role in the stable operation of power grid. As the maintainer of the substation, the on-site staffs work long-term in strong electromagnetic field environment. Therefore, it is necessary to wear the working clothes strictly. In order to strengthen the working clothes wearing circumstance supervision, its better to carry out the real-time supervision on the on-site staffs. In this paper, a video-based working clothes wearing circumstance detection method was put forward. Firstly, we extract characteristics by HOG(Histogram of Oriented Gradient) method and the color spatial distribution compactness presented in this paper. Secondly, the SVM(Support Vector Machine) classifier is trained to realize the substation maintainer detection. Finally, we model the electricity working clothes in the HSV(Hue, Saturation, Value) color space and combine the performance characteristics to get the final results. The experimental results demonstrate that this method has a high accuracy in the substation surveillance video.

Online context-based person re-identification and biometric-based action recognition for service robots

In this paper, we address the problem of person re-identification and action recognition for service robots, which undergoes lack of training dataset for model learning, reduction of feature set discriminative power in changing scenarios, and high complexity of the algorithm computation. An online context-based person re-identification algorithm is proposed, which learns the person model online without pre-collect dataset and adjusts the weight of features according to the context information. An online biometric-based action recognition algorithm is proposed, actions are recognized by simply matching the skeleton vectors extracted from five linkage mechanisms of human body. The proposed algorithms are evaluated on a service robot system, extensive experimental results show that they performs efficiently and effectively in various real-life scenarios.

Human recognition for following robots with a Kinect sensor

In this paper, a human recognition method based on soft biometrics is proposed for the human following robot. Two soft biometric traits (clothes color and body size) are calculated as features of the human. First, the human region detected by the Kinect is segmented to obtain the torso and leg parts of the body. Then the weighted HSV histograms of the body parts are calculated to describe the clothes color information. Body height, arm length and shoulder width values are measured to represent the body size information. Finally, human recognition is implemented by evaluating the similarity among the objects. The effectiveness and robustness of the proposed method is verified by the experiments. And the robot system with the recognition method can recognize and follow a human target reliably.

Body activity interaction for a service robot

Body activity interaction plays an essential role in the field of intelligent robots. In this paper, we design a body activity interaction system for a service robot. In order to realize body activity interaction, the Microsoft Kinect is utilized to capture owner body movements and provide input signals of the action recognition module. Based on the RGBD and skeleton information from Microsoft Kinect, action recognition module can analyze meaning of owner body movements effectively and send orders to service robot quickly. Besides, kinematics model of the service robotic arm is built and the motion planning on the joint space is also executed. Finally, service robot receives orders and conducts robotic arm to implement handshaking or hand waving. Extensive experiments illustrate the correctness and effectiveness of our developed body activity interaction system on the platform of service robot.