Shunsuke Kamijo

3D building model-based pedestrian positioning method using GPS/GLONASS/QZSS and its reliability calculation

Abstract The current low-cost global navigation satellite systems (GNSS) receiver cannot calculate satisfactory positioning results for pedestrian applications in urban areas with dense buildings due to multipath and non-line-of-sight effects. We develop a rectified positioning method using a basic three-dimensional city building model and ray-tracing simulation to mitigate the signal reflection effects. This proposed method is achieved by implementing a particle filter to distribute possible position candidates. The likelihood of each candidate is evaluated based on the similarity between the pseudorange measurement and simulated pseudorange of the candidate. Finally, the expectation of all the candidates is the rectified positioning of the proposed map method. The proposed method will serve as one sensor of an integrated system in the future. For this purpose, we successfully define a positioning accuracy based on the distribution of the candidates and their pseudorange similarity. The real data are recorded at an urban canyon environment in the Chiyoda district of Tokyo using a commercial grade u-blox GNSS receiver. Both static and dynamic tests were performed. With the aid of GLONASS and QZSS, it is shown that the proposed method can achieve a 4.4-m 1σ positioning error in the tested urban canyon area.

NLOS Correction/Exclusion for GNSS Measurement Using RAIM and City Building Models

Currently, global navigation satellite system (GNSS) receivers can provide accurate and reliable positioning service in open-field areas. However, their performance in the downtown areas of cities is still affected by the multipath and none-line-of-sight (NLOS) receptions. This paper proposes a new positioning method using 3D building models and the receiver autonomous integrity monitoring (RAIM) satellite selection method to achieve satisfactory positioning performance in urban area. The 3D building model uses a ray-tracing technique to simulate the line-of-sight (LOS) and NLOS signal travel distance, which is well-known as pseudorange, between the satellite and receiver. The proposed RAIM fault detection and exclusion (FDE) is able to compare the similarity between the raw pseudorange measurement and the simulated pseudorange. The measurement of the satellite will be excluded if the simulated and raw pseudoranges are inconsistent. Because of the assumption of the single reflection in the ray-tracing technique, an inconsistent case indicates it is a double or multiple reflected NLOS signal. According to the experimental results, the RAIM satellite selection technique can reduce by about 8.4% and 36.2% the positioning solutions with large errors (solutions estimated on the wrong side of the road) for the 3D building model method in the middle and deep urban canyon environment, respectively.

Passive Sensor Integration for Vehicle Self-Localization in Urban Traffic Environment.

This research proposes an accurate vehicular positioning system which can achieve lane-level performance in urban canyons. Multiple passive sensors, which include Global Navigation Satellite System (GNSS) receivers, onboard cameras and inertial sensors, are integrated in the proposed system. As the main source for the localization, the GNSS technique suffers from Non-Line-Of-Sight (NLOS) propagation and multipath effects in urban canyons. This paper proposes to employ a novel GNSS positioning technique in the integration. The employed GNSS technique reduces the multipath and NLOS effects by using the 3D building map. In addition, the inertial sensor can describe the vehicle motion, but has a drift problem as time increases. This paper develops vision-based lane detection, which is firstly used for controlling the drift of the inertial sensor. Moreover, the lane keeping and changing behaviors are extracted from the lane detection function, and further reduce the lateral positioning error in the proposed localization system. We evaluate the integrated localization system in the challenging city urban scenario. The experiments demonstrate the proposed method has sub-meter accuracy with respect to mean positioning error.

GPS multipath detection and rectification using 3D maps

Global Navigation Satellite Systems (GNSSs) suffer from the problems of invisible satellites and multipath effect in urban canyons. Many approaches have been employed to eliminate multipath signals in order to reduce positioning errors. Among these, those that consider the geometry of surrounding buildings to improve the evaluation of possible multipath signals have gained most interest. However, such approaches, although successful, require many satellites for positioning after eliminating multipath signals. This study proposes an approach in which the multipath signals themselves are used for positioning error correction. The proposed algorithm evaluates the pseudoranges of the possible multipath signals by referring to the building geometry. The assumed position is estimated by using the pseudoranges and is evaluated by the likelihood of the possible positioning error. The proposed method was verified through field experiments in urban canyons in Tokyo.

Pedestrian dead reckoning for mobile phones through walking and running mode recognition

In this paper, we propose a novel model of stride length estimation for pedestrian dead reckoning (PDR) that allows a PDR system to switch its estimation method according to whether the pedestrian is walking or running. Then, we study the application of the mode switching to a PDR/GPS/map-matching integrated positioning system for mobile phones. The experimental results show that this mode switching makes stride length estimation more adaptive, and improves the total accuracy of positioning.

Vehicle self-localization in urban canyon using 3D map based GPS positioning and vehicle sensors

Precise and robust vehicle localization in the urban canyon is a new challenge arising in the autonomous driving and driver assistance systems. Sensor integration is proposed to realize this target in his paper. Global Positioning System (GPS) has been proven itself reliable for accurate vehicle self-localization in the open sky scenario. However, it suffers from the effect of multipath and Non-Line-Of-Sigh (NLOS) propagation in urban canyon. The paper proposes to estimate vehicle position by using 3-dimensional (3D) map and ray-racing method in order to overcome the problems in urban canyon. The proposed positioning method distributes numbers of positioning candidates around of reference positioning, and then the weighing of the position candidates are evaluated based on the similarity between the simulated pseudorange and the observed pseudorange. In his way, the additional 3D map information is used to reduce the effect of multipath and NLOS. Moreover, the information from vehicle sensors, including motion sensor and rotation sensor, are integrated with he GPS positioning result in a Kalman filer framework. The integration no only smoothens the trajectory of vehicle, but also reduces the positioning error. The experimental results demonstrate the accuracy of our proposed method and is feasibility for autonomous driving.

Customer behavior classification using surveillance camera for marketing

The analysis of customer behavior from surveillance camera is one of the most important open topics for marketing. Traditionally, retailers use the records of cash registers or credit cards to analyze the buying behaviors of customers. However, this information cannot reveal the behaviors of customer when he or she shows interest on the front of the merchandise shelf but does not buy. Those behaviors can be recorded and analyzed by the surveillance camera. We propose a system to classify different customer behaviors on the front of shelf: no interest, viewing, turning body to shelf, touching, picking and returning to shelf and picking and putting into basket, which show customer’s increasing interest to products. In the proposed system, head orientation, body orientation, and arm action, the multiple cues are integrated for the customer behavior recognition. The proposed system discretizes the head and body orientation of customer into 8 directions to estimate whether the customer is looking or turning to the merchandise shelf. Semi-Supervised Learning method is applied to optimize the training dataset and to generate the accurate classifier. In addition, the temporal constraint and the human physical model constraint are considered in joint body and head orientation estimation. As for the arm action recognition, a novel Combined Hand Feature (CHF), which includes hand trajectory, tracking status and the relative position between hand and shopping basket, is proposed to classify different arm actions. The hand tracking is done by an improved particle filter. The CHF is classified by Dynamic Bayesian Network (DBN) to output different types of arm actions. A series of experiments demonstrate effectiveness of the proposed technologies and the performance to the developed system.

Estimation of Pedestrian Pose and Orientation Using on-Board Camera with Histograms of Oriented Gradients Features

Understanding pedestrian behavior, including head and body orientation, is important for a pedestrian safety system. In this paper, we propose an approach that estimates head pose and body orientation by considering two constraints, the pedestrian model constraint between head and body directions and the temporal constraint. In our approach, given an image of pedestrian, image features are extracted and estimates are made of the probabilities of the head position, size and orientation, and the body orientation; these are obtained using a multi-class classifier and tracked by particle filter. We applied two constraints to the particle filter to achieve more accurate estimate. Experiments using real videos from an on-board monocular camera show the effectiveness of our approach.

Probability estimation for pedestrian crossing intention at signalized crosswalks

With the rapid development of the techniques for autonomous driving and ADAS in the last decade, more advanced methods to understand pedestrian behavior are required. Crosswalks at intersections are the one of most hazardous where many accidents between turning-vehicles and pedestrians occur. In this paper, we present a method for estimating the pedestrians intention to cross a signalized crosswalk or stop in front of it. The intention is crucial to not only the collision avoidance but also smooth traffic in the context of autonomous driving by reducing unnecessary risk margins. Regarding the behavioral flow of pedestrian: assessment, decision-making and physical movement, as a stochastic process, we construct a probabilistic model with the Dynamic Bayesian Network. It takes account of not only pedestrian physical states but also contextual information and integrates the relationship among them. By employing the particle filter as a Bayesian filtering framework, the model estimates the pedestrian state from signal information and pedestrian position measurements. Evaluation using experimental data collected in real traffic scene shows that the proposed model has an ability to detect the pedestrian intention to cross a crosswalk even when he/she is far from it.

Towards High-Definition 3D Urban Mapping: Road Feature-Based Registration of Mobile Mapping Systems and Aerial Imagery

Various applications have utilized a mobile mapping system (MMS) as the main 3D urban remote sensing platform. However, the accuracy and precision of the three-dimensional data acquired by an MMS is highly dependent on the performance of the vehicle’s self-localization, which is generally performed by high-end global navigation satellite system (GNSS)/inertial measurement unit (IMU) integration. However, GNSS/IMU positioning quality degrades significantly in dense urban areas with high-rise buildings, which block and reflect the satellite signals. Traditional landmark updating methods, which improve MMS accuracy by measuring ground control points (GCPs) and manually identifying those points in the data, are both labor-intensive and time-consuming. In this paper, we propose a novel and comprehensive framework for automatically georeferencing MMS data by capitalizing on road features extracted from high-resolution aerial surveillance data. The proposed framework has three key steps: (1) extracting road features from the MMS and aerial data; (2) obtaining Gaussian mixture models from the extracted aerial road features; and (3) performing registration of the MMS data to the aerial map using a dynamic sliding window and the normal distribution transform (NDT). The accuracy of the proposed framework is verified using field data, demonstrating that it is a reliable solution for high-precision urban mapping.