Yikai Fang

Robust hand gesture analysis and application in gallery browsing

This paper presents a robust hand gesture analysis method using 3D depth data. Our scheme focuses on accurate hand segmentation by eliminating the negative effect of the forearm part. In the general Human Computer Interaction (HCI) tasks, such an assumption usually holds that the depth of hand is smaller than forearm. Therefore, the precise hand region can be obtained through the fusion of the hand geometric features and the 3D depth information in real-time. Moreover, a robust hand gesture recognition method, which combines the global structure information and the local texture variation, is included in our gesture analysis framework. The elaborate hand segmentation makes the succedent recognition problem much easier and gets more accurate recognition results. Experimental results convincingly show the effectiveness of the proposed gesture analysis strategy. Furthermore, a concrete application scenario, gesture controlled picture gallery browsing, is implemented successfully.

One-against-all frame differences based hand detection for human and mobile interaction

Abstract Hand gesture based human–computer interaction with mobile phones heavily relies on robust hand detection. Because of the limited computational resources of mobile phones, computationally complex algorithms are infeasible. In this paper, we discover that: though traditional frame difference based methods are computationally inexpensive, they have the following problems in many situations: (1) may completely lose the target (we call this as “the phenomenon blind problem”); (2) can detect partial or delayed version of the target (we call this as “the phenomenon delay problem”); and (3) detect the object with many false-alarms (we call this as “the false-alarm problem”). To deal with these problems, we propose an effective and efficient frame-difference algorithm. The proposed algorithm computes one-against-all frame differences and combines them using intersection operators. To determine which pixels in the frame difference are object pixels in interest, we develop a coarse-to-fine binarization procedure. The Hamming distance between LBP (Local Binary Patterns) feature and a local adaptive thresholding scheme are employed in the coarse and fine stages respectively. Experimental results demonstrate the effectiveness and efficiency of the proposed method.

Real-time hand posture analysis based on neural network

In this paper, a modified Neural Gas algorithm is proposed and used to approximate hand topology. As original Neural Gas algorithm is intractable for real-time applications, some optimization such as unnecessary adaption removal and simple learning rate function are introduced to make it applicable for real-time applications. With segmented hand area, the topology representation can be obtained based on neural network. The topology based representation of hand shape will further facilitate both fingertip localization and posture recognition. Experiments show the accuracy and the speed of our method can satisfy realtime requirements of interaction applications, even on mobile devices.

Hand tracking and application in map navigation

Hand tracking is an important component in most practical gesture interaction. Due to background variety, illumination changes and aiticulate shape, its difficult for current conventional vision based methods to obtain accurate results. In this paper, an automatic hand tracking and segmentation method based on depth information is proposed. Hand depth is determined adaptively and then accurate hand region is obtained. In this manner, accurate hand tracking is realized with very low time consumption and regardless of the complex background and various illuminations. Moreover, based on the real-time hand tracking, a robust dynamic gesture recognition strategy is described. Considering the geometric characters, the gestures are classified according to the trajectory fitting and matching with the predefined patterns. Experiments show the effectiveness of the hand tracking and dynamic gesture recognition. Furthermore, a concrete application scenario, gesture controlled map navigation, is implemented with good interaction usability.

Vision Based Hand Posture Recognition

As the prevalence of ubiquitous computing, traditional user interaction approaches with mouse, keyboard and touch pen are not convenient enough for them. In addition, many emerging applications such as augmented reality and interactive entertainments require natural and intuitive interface. Moreover the limited input space on traditional mobile or hand held device leads to encumbered experience with tiny keyboard or touch screen. Hand gesture is frequently used in people’s daily life. It’s also an important component of body languages. So a natural interaction between humans and computing devices can be achieved if hand gestures can be used for communication between human and computing devices. AbsTRACT

Plug&touch: a mobile interaction solution for large display via vision-based hand gesture detection

Smart phones and external large displays are a superior combination for content sharing and multiple person interaction. However, the potential of such a combination is not fully explored due to the lack of effective and portable interaction solutions. To solve this problem, we present a novel mobile solution called Plug&Touch, which is very easy to be set up and enables mobile users to directly and naturally operate on the large display surface. Plug&Touch utilizes vision-based methods to detect the users finger tip and then control user interface. The Local Binary Pattern asymmetric texture difference is proposed to extract the foreground region, and the coarse-to-fine intersection process is adopted to precisely localize the finger tip. Plug&Touch has been implemented on the Symbian3 platform with two defined gestures - click and scroll. Experimental results and user evaluation has shown the practicability of Plug&Touch.