Ruipeng Gao

BatMapper: Acoustic Sensing Based Indoor Floor Plan Construction Using Smartphones

The lack of digital floor plans is a huge obstacle to pervasive indoor location based services (LBS). Recent floor plan construction work crowdsources mobile sensing data from smartphone users for scalability. However, they incur long time (e.g., weeks or months) and tremendous efforts in data collection, and many rely on images thus suffering technical and privacy limitations. In this paper, we propose BatMapper, which explores a previously untapped sensing modality -- acoustics -- for fast, fine grained and low cost floor plan construction. We design sound signals suitable for heterogeneous microphones on commodity smartphones, and acoustic signal processing techniques to produce accurate distance measurements to nearby objects. We further develop robust probabilistic echo-object association, recursive outlier removal and probabilistic resampling algorithms to identify the correspondence between distances and objects, thus the geometry of corridors and rooms. We compensate minute hand sway movements to identify small surface recessions, thus detecting doors automatically. Experiments in real buildings show BatMapper achieves 1-2cm distance accuracy in ranges up around 4m; a 2-3 minute walk generates fine grained corridor shapes, detects doors at 92% precision and 1~2m location error at 90-percentile; and tens of seconds of measurement gestures produce room geometry with errors <0.3m at 80-percentile, at 1-2 orders of magnitude less data amounts and user efforts.

Knitter: Fast, resilient single-user indoor floor plan construction

Lacking of floor plans is a fundamental obstacle to ubiquitous indoor location-based services. Recent work have made significant progress to accuracy, but they largely rely on slow crowdsensing that may take weeks or even months to collect enough data. In this paper, we propose Knitter that can generate accurate floor maps by a single random users one hour data collection efforts. Knitter extracts high quality floor layout information from single images, calibrates user trajectories and filters outliers. It uses a multi-hypothesis map fusion framework that updates landmark positions/orientations and accessible areas incrementally according to evidences from each measurement. Our experiments on 3 different large buildings and 30+ users show that Knitter produces correct map topology, and 90-percentile landmark location and orientation errors of 3 ∼ 5m and 4 ∼ 6°, comparable to the state-of-the-art at more than 20× speed up: data collection can finish in about one hour even by a novice user trained just a few minutes.

BatTracker: High Precision Infrastructure-free Mobile Device Tracking in Indoor Environments

Continuous tracking of the device location in 3D space is a popular form of user input, especially for virtual/augmented reality (VR/AR), video games and health rehabilitation. Conventional inertial based approaches are well known for inaccuracy caused by large error drifts. Computer vision approaches can produce accuracy tracking but have privacy concerns and are subject to lighting conditions and computation complexity. Recent work exploits accurate acoustic distance measurements for high precision tracking. However, they require additional hardware (e.g., multiple external speakers), which adds to the costs and installation efforts, thus limiting the convenience and usability. In this paper, we propose BatTracker, which incorporates inertial and acoustic data for robust, high precision and infrastructure-free tracking in indoor environments. BatTracker leverages echoes from nearby objects and uses distance measurements from them to correct error accumulation in inertial based device position prediction. It incorporates Doppler shifts and echo amplitudes to reliably identify the association between echoes and objects despite noisy signals from multi-path reflection and cluttered environment. A probabilistic algorithm creates, prunes and evolves multiple hypotheses based on measurement evidences to accommodate uncertainty in device position. Experiments in real environments show that BatTracker can track a mobile devices movements in 3D space at sub-cm level accuracy, comparable to the state-of-the-art infrastructure based approaches, while eliminating the needs of any additional hardware.

Demo: Acoustic Sensing Based Indoor Floor Plan Construction Using Smartphones

This demo presents BatMapper, an acoustics sensing technology for fast, fine-grained and low cost floor plan construction. BatMapper operates by emitting sound signal and capturing its reflections by two microphones on smartphones. We develop robust probabilistic echo-object association and outlier removal algorithms to identify the correspondence between distances and objects, thus the geometry of corridors. We compensate minute hand sway movements to identify small surface recessions, thus detecting doors automatically. Additionally, we leverage structure cues in indoor environments for user trace calibration. The demo will enable any person to hold the smartphone and walk along a corridor to map the corridor shape and detect doors in real-time.

Introduction of Indoor Map Construction

We describe the motivation and background of map construction for ubiquitous indoor location-based services, and then give an overview of this book and present how it is organized in the following chapters.

Smartphone-Based Indoor Map Construction

We describe the motivation and background of map construction for ubiquitous indoor location-based services, and then give an overview of this book and present how it is organized in the following chapters.

Indoor Map Construction via Mobile Crowdsensing

The lack of indoor maps is a critical reason behind the current sporadic availability of indoor localization service. Service providers have to go through effort-intensive and time-consuming business negotiations with building operators, or hire dedicated personnel to gather such data. In this chapter, we propose Jigsaw, a floor plan reconstruction system that leverages crowdsensed data from mobile users. It extracts the position, size, and orientation information of individual landmark objects from images taken by users. It also obtains the spatial relation between adjacent landmark objects from inertial sensor data, and then computes the coordinates and orientations of these objects on an initial floor plan. By combining user mobility traces and locations where images are taken, it produces complete floor plans with hallway connectivity, room sizes, and shapes. It also identifies different types of connection areas (e.g., escalators, stairs) between stories, and employs a refinement algorithm to correct detection errors. Our experiments on three stories of two large shopping malls show that the 90-percentile errors of positions and orientations of landmark objects are about 1 \(\sim \) 2 m and 5 \(\sim \) 9\(^\circ \), while the hallway connectivity and connection areas between stories are 100% correct.

Indoor Localization by Photo-Taking of the Environment

Mainstream indoor localization technologies rely on RF signatures that require extensive human efforts to measure and periodically recalibrate signatures. The progress to ubiquitous localization remains slow. In this chapter, we explore Sextant, an alternative approach that leverages environmental reference objects such as store logos. A user uses a smartphone to obtain relative position measurements to such static reference objects for the system to triangulate the user location. Sextant leverages image matching algorithms to automatically identify the chosen reference objects by photo-taking, and we propose two methods to systematically address image matching mistakes that cause large localization errors. We formulate the benchmark image selection problem, prove its NP-completeness, and propose a heuristic algorithm to solve it. We also propose a couple of geographical constraints to further infer unknown reference objects. To enable fast deployment, we propose a lightweight site survey method for service providers to quickly estimate the coordinates of reference objects. Extensive experiments have shown that Sextant prototype achieves 2–5 m accuracy at 80-percentile, comparable to the industry state of the art, while covering a \(150\times 75\) m mall and \(300\times 200\) m train station requires a one-time investment of only 2–3 man-hours from service providers.

Incremental Indoor Map Construction with a Single User

Lacking of floor plans is a fundamental obstacle to ubiquitous indoor location-based services. Recent work have made significant progress to accuracy, but they largely rely on slow crowdsensing that may take weeks or even months to collect enough data. In this chapter, we propose Knitter that can generate accurate floor maps by a single random user’s one-hour data collection efforts, and demonstrate how such maps can be used for indoor navigation. Knitter extracts high-quality floor layout information from single images, calibrates user trajectories, and filters outliers. It uses a multi-hypothesis map fusion framework that updates landmark positions/orientations and accessible areas incrementally according to evidences from each measurement. Our experiments on three different large buildings and 30+ users show that Knitter produces correct map topology, and 90-percentile landmark location and orientation errors of \(3\sim 5\,\mathrm{m}\) and \(4\sim 6^\circ \), comparable to the state of the art at more than \(20\times \) speed up: data collection can finish in about one hour even by a novice user trained just a few minutes.

VeMap: Indoor Road Map Construction via Smartphone-Based Vehicle Tracking

Since GPS signal is not applicable indoors, vehicle tracking has proven a hassle in underground parking structures. Recent solutions highly rely on floor map to constraint inertial sensors noises. In this paper, we propose VeMap, a road map construction system using only smartphones inside vehicles. It saves effort-intensive and time-consuming business negotiations with building operators, and expensive personnel cost to gather such data. It fuses multiple sensors to calibrate inertial noises, and uses Dynamic Time Warping to align multiple trajectories. We represent the floor plan with occupancy grid mapping, and explore a vision-mobile joint algorithm to extract its skeleton and form the road map. VeMap is tested in a 250mx90m parking structure, and it can be directly used for driving navigation to free parking spaces.