Furkan Kıraç

Real time hand pose estimation using depth sensors

This paper describes a depth image based real-time skeleton fitting algorithm for the hand, using an object recognition by parts approach, and the use of this hand modeler in an American Sign Language (ASL) digit recognition application. In particular, we created a realistic 3D hand model that represents the hand with 21 different parts. Random decision forests (RDF) are trained on synthetic depth images generated by animating the hand model, which are then used to perform per pixel classification and assign each pixel to a hand part. The classification results are fed into a local mode finding algorithm to estimate the joint locations for the hand skeleton. The system can process depth images retrieved from Kinect in real-time at 30 fps. As an application of the system, we also describe a support vector machine (SVM) based recognition module for the ten digits of ASL based on our method, which attains a recognition rate of 99.9% on live depth images in real-time1.

A tabu search algorithm for parallel machine total tardiness problem

In this study, we consider the problem of scheduling a set of independent jobs with sequence dependent setups on a set of uniform parallel machines such that total tardiness is minimized. Jobs have nonidentical due dates and arrival times. A tabu search (TS) approach is employed to attack this complex problem. In order to obtain a robust search mechanism, several key components of TS such as candidate list strategies, tabu classifications, tabu tenure and intensification/diversification strategies are investigated. Alternative approaches to each of these issues are developed and extensively tested on a set of problems obtained from the literature. The results obtained are considerably better than those reported previously and constitute the best solutions known for the benchmark problems as to date.

A tabu search algorithm for the single machine total weighted tardiness problem

In this study, a tabu search (TS) approach to the single machine total weighted tardiness problem (SMTWT) is presented. The problem consists of a set of independent jobs with distinct processing times, weights and due dates to be scheduled on a single machine to minimize total weighted tardiness. The theoretical foundation of single machine scheduling with due date related objectives reveal that the problem is NP-hard, rendering it a challenging area for meta-heuristic approaches. This paper presents a totally deterministic TS algorithm with a hybrid neighborhood and dynamic tenure structure, and investigates the strength of several candidate list strategies based on problem specific characteristics in increasing the efficiency of the search. The proposed TS approach yields very high quality results for a set of benchmark problems obtained from the literature. � 2005 Elsevier B.V. All rights reserved.

Randomized decision forests for static and dynamic hand shape classification

This paper proposes a novel algorithm to perform hand shape classification using depth sensors, without relying on color or temporal information. Hence, the system is independent of lighting conditions and does not need a hand registration step. The proposed method uses randomized classification forests (RDF) to assign class labels to each pixel on a depth image, and the final class label is determined by voting. This method is shown to achieve 97.8% success rate on an American Sign Language (ASL) dataset consisting of 65k images collected from five subjects with a depth sensor. More experiments are conducted on a subset of the ChaLearn Gesture Dataset, consisting of a lexicon with static and dynamic hand shapes. The hands are found using motion cues and cropped using depth information, with a precision rate of 87.88% when there are multiple gestures, and 94.35% when there is a single gesture in the sample. The hand shape classification success rate is 94.74% on a small subset of nine gestures corresponding to a single lexicon. The success rate is 74.3% for the leave-one-subject-out scheme, and 67.14% when training is conducted on an external dataset consisting of the same gestures. The method runs on the CPU in real-time, and is capable of running on the GPU for further increase in speed.

Hierarchically constrained 3D hand pose estimation using regression forests from single frame depth data

We apply Random Forests for regression (RDF-R) to 3D Hand Pose Estimation.RDF-R is generalized by hierarchical mode selection using constraints (RDF-R+).We test Classification Forests (RDF-C), RDF-R and RDF-R+ with 4 different datasets.The proposed method, RDF-R+ outperforms RDF-C and RDF-R. The emergence of inexpensive 2.5D depth cameras has enabled the extraction of the articulated human body pose. However, human hand skeleton extraction still stays as a challenging problem since the hand contains as many joints as the human body model. The small size of the hand also makes the problem more challenging due to resolution limits of the depth cameras. Moreover, hand poses suffer from self-occlusion which is considerably less likely in a body pose. This paper describes a scheme for extracting the hand skeleton using random regression forests in real-time that is robust to self- occlusion and low resolution of the depth camera. In addition to that, the proposed algorithm can estimate the joint positions even if all of the pixels related to a joint are out of the camera frame. The performance of the new method is compared to the random classification forests based method in the literature. Moreover, the performance of the joint estimation is further improved using a novel hierarchical mode selection algorithm that makes use of constraints imposed by the skeleton geometry. The performance of the proposed algorithm is tested on datasets containing synthetic and real data, where self-occlusion is frequently encountered. The new algorithm which runs in real time using a single depth image is shown to outperform previous methods.

Real-time pose tracking based on a 3D skeletal model using multiple cameras

In this work a robust algorithm for tracking an object and its pose is developed when its 3D skeletal model is known. Proposed system works in real-time using particle filters on low dimensional manifolds captured by Gaussian process latent variable model (GPLVM). In this experimental work we rendered our ground truth silhouettes for generating the perfecly segmented data. A synthetically generated walking sequence from recorded motion capture data is tracked in real-time using a standard personal computer.

3D hand pose estimation and classification using depth sensors

This paper describes our method to fit a 3D skeleton to the human hand using depth images. The human hand is represented by a 3D skeleton with 21 parts. This model is used to generate synthetic depth images, that are used to train Random Decision Forests (RDF), which are used to assign each pixel to a hand part. Mean-shift method is used on the classification results and joint locations are estimated. The system can run in real time at 30 fps on Kinect depth images. We use this method and Support Vector Machines for classification and obtain 99.9% recognition rate on the American Sign Language (ASL) digit recognition problem.

Depth image based 3D hand pose estimation framework

Real-time 3D motion capture for the human hand opens many avenues for HCI. This work describes our framework for fitting a 3D skeleton to the human hand using depth images. We represent a human hand by a 3D skeleton with 15 joints. Using this model, various synthetic depth images are generated. Random Decision Forests (RDF) are trained and used to assign each pixel to a hand part. A mean-shift method is used for estimating joint locations using pixel classification results. Our system runs in real time at 30 fps on Kinect depth images.

Human hand tracking in image sequence using particle filter

In this work we have developed a new algorithm for the purpose of tracking a human hand in a video sequence acquired by a standard color video camera. The proposed system works on previously segmented hand images . Segmentation is achieved with a skin color filter. However, when more than one skin-colored region is present in the image, tracking becomes difficult. In order to satisfy both real-time and successful tracking at the same time, a model based particle filter has been preferred. An ellipse is selected as the shape model of a human hand. Despite the simplicity of the used model, the tracking of human hand shape in video imagery yields very good results.