2019 11th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC) | 2019
A Precise Fingertip Tracking Algorithm Based on 3D Motion Mapping for HoloLens
Abstract
Fingertip s motion is an important input channel of Human-Computer-Interaction in Augmented Reality, but HoloLens depth-image is not accessible nowadays. In this paper, a precise and fast fingertip tracking algorithm is proposed. Firstly, we get 2D fingertip s position from monocular camera video stream by segmentation method. Then, we setup a spherical coordinate system according to the hierarchical finger-hand-arm s motion model to project the fingertip s motion in 2D image to 3D space. Meanwhile, a composite Kalman motion model is designed for 3D tracking prediction and smoothing. With the 3D Kalman filter s help, high-precise fingertip motion is projected back to the 2D space, to promote the fingertip detection of the next state. The fingertip detection is performed in a small neighbourhood of the predicted position. Experiments show that the accuracy and performance are both promoted than traditional fingertip detection method. The algorithm is applied in HoloLens AR gesture interaction system and can provide accurate and fast gesture tracking.