An improved colour binary descriptor algorithm for mobile\naugmented reality

Abstract

The incorporation of both virtual content and real world in augmented reality (AR) allows real-time engagement with the virtual objects. The selection of an appropriate tracking algorithm is important to optimise the performance of mobile AR applications given the limited processing capabilities and memories of mobile devices like smartphones. Tracking in AR consists of four essential components, namely detector, descriptor, matcher, and pose estimator. Since a descriptor substantially affects the overall performance of a mobile AR application, it must have short computational time and remains invariant to scale, rotation, and lighting changes. Studies have proposed Fast Retina Keypoint (FREAK) descriptor as the most suitable descriptor for mobile AR applications. Unlike other greyscale descriptors, FREAK has shorter computational time and is less likely to be affected by scale and rotation changes. However, it overlooks the vital colour space information. Focusing on enhancing the efficiency and robustness of FREAK, this study proposed the use of CRH-FREAK (RGB\u2009+\u2009HSV) descriptor and applied the vertical concatenation technique that combined all extracted keypoints vertically. The robustness of the proposed descriptors against scale, rotation, and lighting changes was verified using Mikolajczyk and Amsterdam Library of Object Images (ALOI) datasets. The developed CRH-FREAK descriptors used six colour spaces to describe the keypoints, which made them slower than the original FREAK. However, the size reduction of CRH-FREAK from 512 bits to 128 bits in this study successfully reduced the computational time to 29.49\xa0ms, which was found comparable to the original FREAK. The improved efficiency and robustness of a 128-bit CRH-FREAK descriptor benefit the future development of mobile AR applications that remain invariant to scale, rotation, and lighting changes.