Hanjiang Xiong

Activity Recognition and Semantic Description for Indoor Mobile Localization

As a result of the rapid development of smartphone-based indoor localization technology, location-based services in indoor spaces have become a topic of interest. However, to date, the rich data resulting from indoor localization and navigation applications have not been fully exploited, which is significant for trajectory correction and advanced indoor map information extraction. In this paper, an integrated location acquisition method utilizing activity recognition and semantic information extraction is proposed for indoor mobile localization. The location acquisition method combines pedestrian dead reckoning (PDR), human activity recognition (HAR) and landmarks to acquire accurate indoor localization information. Considering the problem of initial position determination, a hidden Markov model (HMM) is utilized to infer the user’s initial position. To provide an improved service for further applications, the landmarks are further assigned semantic descriptions by detecting the user’s activities. The experiments conducted in this study confirm that a high degree of accuracy for a user’s indoor location can be obtained. Furthermore, the semantic information of a user’s trajectories can be extracted, which is extremely useful for further research into indoor location applications.

An improved ANUDEM method combining topographic correction and DEM interpolation

Void filling and anomaly replacement in conjunction with auxiliary sources of data have been widely used to improve the quality of existing problematic high-resolution digital elevation models. However, the traditional interpolation methods used for this purpose have always failed to eliminate the discrepancies between different data-sets. In this paper, an improved ANUDEM method is presented for DEM interpolation, which incorporates the idea of topographic correction using high correlation of topological structure between contour lines (CLs) from multi-scale digital elevation models (DEM). Firstly, the terrain topological structure is extracted from the CLs of a low-resolution DEM. The topographic surface correction is then undertaken based on the extracted structure, which recovers the topographic information of the sharp depressions and eminences to fit the high-resolution representation. Finally, the breaklines of the terrain surface are distilled and integrated into the denser DEM generation. The experiments undertaken confirmed the superiority of the proposed method over the other DEM interpolation methods. It is shown that the proposed method can provide results with a higher accuracy, as well as a better visual quality.

An Indoor Scene Recognition-Based 3D Registration Mechanism for Real-Time AR-GIS Visualization in Mobile Applications

Mobile Augmented Reality (MAR) systems are becoming ideal platforms for visualization, permitting users to better comprehend and interact with spatial information. Subsequently, this technological development, in turn, has prompted efforts to enhance mechanisms for registering virtual objects in real world contexts. Most existing AR 3D Registration techniques lack the scene recognition capabilities needed to describe accurately the positioning of virtual objects in scenes representing reality. Moreover, the application of such registration methods in indoor AR-GIS systems is further impeded by the limited capacity of these systems to detect the geometry and semantic information in indoor environments. In this paper, we propose a novel method for fusing virtual objects and indoor scenes, based on indoor scene recognition technology. To accomplish scene fusion in AR-GIS, we first detect key points in reference images. Then, we perform interior layout extraction using a Fully Connected Networks (FCN) algorithm to acquire layout coordinate points for the tracking targets. We detect and recognize the target scene in a video frame image to track targets and estimate the camera pose. In this method, virtual 3D objects are fused precisely to a real scene, according to the camera pose and the previously extracted layout coordinate points. Our results demonstrate that this approach enables accurate fusion of virtual objects with representations of real world indoor environments. Based on this fusion technique, users can better grasp virtual three-dimensional representations on an AR-GIS platform.

A Novel Semantic Matching Method for Indoor Trajectory Tracking

The rapid development of smartphone sensors has provided rich indoor pedestrian trajectory data for indoor location-based applications. To improve the quality of these collected trajectory data, map matching methods are widely used to correct trajectories. However, these existing matching methods usually cannot achieve satisfactory accuracy and efficiency and have difficulty in exploiting the rich information contained in the obtained trajectory data. In this study, we proposed a novel semantic matching method for indoor pedestrian trajectory tracking. Similar to our previous work, pedestrian dead reckoning (PDR) and human activity recognition (HAR) are used to obtain the raw user trajectory data and the corresponding semantic information involved in the trajectory, respectively. To improve the accuracy and efficiency for user trajectory tracking, a semantic-rich indoor link-node model is then constructed based on the input floor plan, in which navigation-related semantics are extracted and formalized for the following trajectory matching. PDR and HAR are further utilized to segment the trajectory and infer the semantics (e.g., “Turn left”, “Turn right”, and “Go straight”). Finally, the inferred semantic information is matched with the semantic-rich indoor link-node model to derive the correct user trajectory. To accelerate the matching process, the semantics inferred from the trajectory are also assigned weights according to their relative importance. The experiments confirm that the proposed method achieves accurate trajectory tracking results while guaranteeing a high matching efficiency. In addition, the resulting semantic information has great application potential in further indoor location-based services.

Image-Based Localization Aided Indoor Pedestrian Trajectory Estimation Using Smartphones

Accurately determining pedestrian location in indoor environments using consumer smartphones is a significant step in the development of ubiquitous localization services. Many different map-matching methods have been combined with pedestrian dead reckoning (PDR) to achieve low-cost and bias-free pedestrian tracking. However, this works only in areas with dense map constraints and the error accumulates in open areas. In order to achieve reliable localization without map constraints, an improved image-based localization aided pedestrian trajectory estimation method is proposed in this paper. The image-based localization recovers the pose of the camera from the 2D-3D correspondences between the 2D image positions and the 3D points of the scene model, previously reconstructed by a structure-from-motion (SfM) pipeline. This enables us to determine the initial location and eliminate the accumulative error of PDR when an image is successfully registered. However, the image is not always registered since the traditional 2D-to-3D matching rejects more and more correct matches when the scene becomes large. We thus adopt a robust image registration strategy that recovers initially unregistered images by integrating 3D-to-2D search. In the process, the visibility and co-visibility information is adopted to improve the efficiency when searching for the correspondences from both sides. The performance of the proposed method was evaluated through several experiments and the results demonstrate that it can offer highly acceptable pedestrian localization results in long-term tracking, with an error of only 0.56 m, without the need for dedicated infrastructures.

Indoor scene texturing based on single mobile phone images and 3D model fusion

ABSTRACT Realistic texture mapping and coherent up-to-date rendering is one of the most important issues in indoor 3-D modelling. However, existing texturing approaches are usually performed manually during the modelling process, and cannot accommodate changes in indoor environments occurring after the model was created, resulting in out-dated and misleading texture rendering. In this study, a structured learning-based texture mapping method is proposed for automatic mapping a single still photo from a mobile phone onto an already-constructed indoor 3-D model. The up-to-date texture is captured using a smart phone, and the indoor structural layout is extracted by incorporating per-pixel segmentation in the FCN algorithm and the line constraints into a structured learning algorithm. This enables real-time texture mapping according to parts of the model, based on the structural layout. Furthermore, the rough camera pose is estimated by pedestrian dead reckoning (PDR) and map information to determine where to map the texture. The experimental results presented in this paper demonstrate that our approach can achieve accurate fusion of 3-D triangular meshes with 2-D single images, achieving low-cost and automatic indoor texture updating. Based on this fusion approach, users can have a better experience in virtual indoor3-D applications.

Robust Indoor Mobile Localization with a Semantic Augmented Route Network Graph

In recent years, using smartphones to determine pedestrian locations in indoor environments is an extensively promising technique for improving context-aware applications. However, the applicability and accuracy of the conventional approaches are still limited due to infrastructure-dependence, and there is seldom consideration of the semantic information inherently existing in maps. In this paper, a semantically-constrained low-complexity sensor fusion approach is proposed for the estimation of the user trajectory within the framework of the smartphone-based indoor pedestrian localization, which takes into account the semantic information of indoor space and its compatibility with user motions. The user trajectory is established by pedestrian dead reckoning (PDR) from the mobile inertial sensors, in which the proposed semantic augmented route network graph with adaptive edge length is utilized to provide semantic constraint for the trajectory calibration using a particle filter algorithm. The merit of the proposed method is that it not only exploits the knowledge of the indoor space topology, but also exhausts the rich semantic information and the user motion in a specific indoor space for PDR accumulation error elimination, and can extend the applicability for diverse pedestrian step length modes. Two experiments are conducted in the real indoor environment to verify of the proposed approach. The results confirmed that the proposed method can achieve highly acceptable pedestrian localization results using only the accelerometer and gyroscope embedded in the phones, while maintaining an enhanced accuracy of 1.23 m, with the indoor semantic information attached to each pedestrian’s motion.