Jie Nie

Robust skin detection in real-world images

Proposed scheme incorporates color property, texture property and region property.Robust skin seeds are acquired by combining color property and texture property.SCTGC incorporates color property, texture property and region property. Human skin detection in images is desirable in many practical applications, e.g., human-computer interaction and adult-content filtering. However, existing methods are mainly suffer from confusing backgrounds in real-world images. In this paper, we try to address this issue by exploring and combining several human skin properties, i.e. color property, texture property and region property. First, images are divided into superpixels, and robust skin seeds and background seeds are acquired through color property and texture property of skin. Then we combining color, region and texture properties of skin by proposing a novel skin color and texture based graph cuts (SCTGC) to acquire the final skin detection results. Comprehensive and comparative experiments show that the proposed method achieves promising performance and outperforms many state-of-the-art methods over publicly available challenging datasets with a great part of hard images.

Is Your First Impression Reliable? Trustworthy Analysis Using Facial Traits in Portraits

As a basic human quality, trustworthiness plays an important role in social communications. In this paper, we proposed a novel concept to predict people’s trustworthiness at first sight using facial traits. Firstly, personality-toward traits were designed from psychology, including permanent traits and transient traits. Then, a mixture of feature descriptors consisting of Histogram of Gradients (HOG), Local Binary Patterns (LBP) and geometrical descriptions were adopted to describe personality traits. Finally, we trained the personality traits by LibSVM to determine trustworthiness of a person using portrait. Experiments demonstrated the effectiveness of our method by improving the precision by 33.60%, recall by 20.33% and F1-measure by 25.63% when determining whether a person is trustworthy or not comparing to a baseline method. Feature contribution analysis was applied to deeply unveil the correspondence between features and personality. Demonstration showed visual patterns in portrait collages of trustworthy people that further proved effectiveness of our method.

Human body segmentation based on shape constraint

Human body segmentation is essential for many practical applications, e.g., video surveillance analysis in intelligent urban. However, existing methods mainly suffer from various human poses. In this paper, we try to address this issue by introducing human shape constraint. First, human pose estimation is performed, and locations of human body parts are determined. Contrast to the previous work, we just use the human body parts with high precision. Then we combines the star convexity and the human body parts’ locations as shape constraint. The final segmentation results are acquired through the optimization step. Comprehensive and comparative experimental results demonstrate that the proposed method achieves promising performance and outperforms many state-of-the-art methods over publicly available challenging datasets.

Social Media Profiler: Inferring Your Social Media Personality from Visual Attributes in Portrait

In this paper, we introduce an interesting but challenging problem: how to infer social media personality from portrait. To address this problem, we jointly consider social media content and behavior information. Specifically, first, we represent social media personality as a reflection in accordance with user behaviors in social media. Second, by means of clustering, people are divided into eight groups and labeled with different personality types. Upon regression analysis, discriminative visual attributes for personality classification are determined. Third, low-level features of selected visual attributes are trained to predict personality from given portrait. To evaluate the proposed method, we collect images of people from the internet and the behaviors of these people from their micro-blog. Comprehensive experiments demonstrate that the proposed method can achieve significant performance gain over the existing method.

Thinking like psychologist: Learning to predict personality by using features from portrait and social media

It is interesting but challenging to infer peoples personality from multimedia data. This paper resolving this problem by jointly considered both human appearances from portrait image and behavior representative data from social media, where features are designed and selected separately inspired by psychology theory. Best features are selected for four personality categories respectively depending on classification performance. Experiment demonstrated that: by adding social media features, our method achieves an average improvement of 24.95% in true positive rate than the state of the art. Meanwhile, by introducing features inspired by psychology theory, our portrait features outperformed the baseline method by increasing 11.69% in true positive rate.

Exploring Relationship Between Face and Trustworthy Impression Using Mid-level Facial Features

When people look at a face, they always build an affective subconscious impression of the person which is very useful information in social contact. Exploring relationship between facial appearance in portraits and personality impression is an interesting and challenging issue in multimedia area. In this paper, a novel method which can build relationship between facial appearance and personality impression is proposed. Low-level visual features are extracted on the defined face regions designed from psychology at first. Then, to alleviate the semantic gap between the low-level features and high-level affective features, mid-level feature set are built through clustering method. Finally, classification model is trained using our dataset. Comprehensive experiments demonstrate the effectiveness of our method by improving 26.24i¾?% in F1-measure and 54.28i¾?% in recall under similar precision comparing to state-of-the-art works. Evaluation of different mid-level feature combinations further illustrates the promising of the proposed method.

Hierarchical Activity Recognition Using Smart Watches and RGB-Depth Cameras

Human activity recognition is important for healthcare and lifestyle evaluation. In this paper, a novel method for activity recognition by jointly considering motion sensor data recorded by wearable smart watches and image data captured by RGB-Depth (RGB-D) cameras is presented. A normalized cross correlation based mapping method is implemented to establish association between motion sensor data with corresponding image data from the same person in multi-person situations. Further, to improve the performance and accuracy of recognition, a hierarchical structure embedded with an automatic group selection method is proposed. Through this method, if the number of activities to be classified is changed, the structure will be changed correspondingly without interaction. Our comparative experiments against the single data source and single layer methods have shown that our method is more accurate and robust.

Online multi-objective optimization for live video forwarding across video data centers

An online live video forwarding approach in video data centers.Multi-objective optimization of forwarding delay, resource cost, and scalability.Lowest time complexity and high efficiency on the real-world scenario. The proliferation of video surveillance has led to surveillance video forwarding services becoming a basic server in video data centers. End users in diverse locations require live video streams from the IP cameras through the inter-connected video data centers. Consequently, the resource scheduler, which is set up to assign the resources of the video data centers to each arriving end user, is in urgent need of achieving the global optimal resource cost and forwarding delay. In this paper, we propose a multi-objective resource provisioning (MORP) approach to minimize the resource provisioning cost during live video forwarding. Different from existed works, the MORP optimizes the resource provisioning cost from both the resource cost and forwarding delay. Moreover, as an approximate optimal approach, MORP adaptively assigns the proper media servers among video data centers, and connects these media servers together through network connections to provide system scalability and connectivity. Finally, we prove that the computational complexity of our online approach is only O(log(|U|)) (|U| is the number of arrival end users). The comprehensive evaluations show that our approach not only significantly reduces the resource provisioning cost, but also has a considerably shorter computational delay compared to the benchmark approaches.

How to record the amount of exercise automatically? A general real-time recognition and counting approach for repetitive activities

Exercise is considered as an effective mean against overweight and obesity-related diseases. In this paper, a real-time activity recognition and counting approach is proposed to evaluate amount of exercise only using a wearable smart watch. First, accelerometer and gyroscope data are collected to extract efficient features. Then Support Vector Machine classifiers are trained to recognize nine common exercise activities in real time. In order to measure the frequency of repetitive activity, a general activity counting algorithm based on gyroscope is proposed which is applicable for different types of activity. Various activities can be counted uninterruptedly using the proposed general method without frequently changing algorithms. Through experiments, it is demonstrated that the extracted features are efficient for real time exercise activity recognition. Moreover, our comparative experiments have shown that our counting approach is more accurate than other products on the market.

Finding suits in images of people in unconstrained environments

The main direction of human body is introduced in order to cope with various human poses.We propose three novel kinds of features, i.e., color features, shape features and statistical features.We try to address the suit detection issue for images of people in unconstrained environments. Clothing style analysis is a critical step for understanding images of people. To automatically identify the style of clothing that people wear is a challenging task due to various poses of person and large variations for even the same clothing category. Suit as one of the clothing style is a key element in many important activities. In this paper, we propose a novel suits detection method for images of people in unconstrained environments. In order to cope with various human poses, human pose estimation is incorporated. By analyzing the style of clothing, we propose the color features, shape features and statistical features for suits detection. Experiments with four popular classifiers have been conducted to demonstrate that the proposed features are effective and robust. Comparative experiments with Bag of Words (BoW) method demonstrate that the proposed features are superior to BoW which is a popular method for object detection. The proposed method has achieved promising performance over our dataset, which is a challenging web image set with various human poses and diverse styles of clothing.