Harlan Hile

Landmark-based pedestrian navigation from collections of geotagged photos

Mobile phones are an attractive platform for landmark-based pedestrian navigation systems. To be practical, such a system must be able to automatically generate lightweight directions that can be displayed on these mobile devices. We present a system that leverages an online collection of geotagged photographs to automatically generate navigational instructions. These are presented to the user as a sequence of images of landmarks augmented with directional instructions. Both the landmark selection and image augmentation are done automatically. We present a user study that indicates these generated directions are beneficial to users and suggest areas for future improvement.

Positioning and Orientation in Indoor Environments Using Camera Phones

Location technologies provide a way to associate information to a users location. The authors developed a system that processes a cell-phone camera image and matches detected landmarks from the image to a building. The system calculates camera location and dynamically overlays information directly on the cell phone image.

Landmark-Based Pedestrian Navigation with Enhanced Spatial Reasoning

Computer vision techniques can enhance landmark-based navigation by better utilizing online photo collections. We use spatial reasoning to compute camera poses, which are then registered to the world using GPS information extracted from the image tags. Computed camera pose is used to augment the images with navigational arrows that fit the environment. We develop a system to use high-level reasoning to influence the selection of landmarks along a navigation path, and lower-level reasoning to select appropriate images of those landmarks. We also utilize an image matching pipeline based on robust local descriptors to give users of the system the ability to capture an image and receive navigational instructions overlaid on their current context. These enhancements to our previous navigation system produce a more natural navigation plan and more understandable images in a fully automatic way.

Informing the design of an automated wayfinding system for individuals with cognitive impairments

Individuals with cognitive impairments are often prevented from independently living, working, and fully participating in their community due to wayfinding concerns. We conducted two user studies of a mobile wayfinding aid designed to support such individuals. The first study examined usability issues related to wayfinding outdoors. The results were positive overall, but showed that the directions we used were at times too low-level, requiring strict adherence to the route and therefore highly precise message timing. The second study examined the use of landmarks to provide directions at a higher-level, as a way to overcome the limitations of the directions we were using. We found that certain types of landmark-based directions were significantly easier to follow, but individual performance varied across most direction types. The findings from both studies show that individuals with cognitive impairments would benefit from a wayfinding system that is capable of supporting customizable and adaptable direction selection.

Customizing directions in an automated wayfinding system for individuals with cognitive impairment

Individuals with cognitive impairments would prefer to live independently, however issues in wayfinding prevent many from fully living, working, and participating in their community. Our research has focused on designing, prototyping, and evaluating a mobile wayfinding system to aid such individuals. Building on the feedback gathered from potential users, we have implemented the systems automated direction selection functionality. Using a decision-theoretic approach, we believe we can create better wayfinding experience that assists users to reach their destination more intuitively than traditional navigation systems. This paper describes the system and results from a study using system-generated directions that inform us of key customization factors that would provide improved wayfinding assistance for individual users.

Visual Navigation for Mobile Devices

This article presents the integration of an improved camera pose recovery method into a landmark-based visual navigation system for mobile devices.

Creating compact architectural models by geo-registering image collections

We present a method for automatically constructing compact, photo-realistic architectural 3D models. This method uses simple 3D building outlines obtained from existing GIS databases to bootstrap reconstruction and works with both structured and unstructured image datasets. We propose an optimal view-selection algorithm for selecting a small set of views for texture mapping that best describe the structure, while minimizing warping and stitching artifacts, and producing a consistent visual representation. The proposed method is fully automatic and can process large structured datasets in close to real-time, making it suitable for large scale urban modeling and 3D map construction.

Microbiology Tray and Pipette Tracking as a Proactive Tangible User Interface

Many work environments can benefit from integrated computing devices to provide information to users, record users’ actions, and prompt users about the next steps to take in a procedure. We focus on the cell biology laboratory, where previous work on the Labscape project has provided a framework to organize experiment plans and store data. Currently developed sensor systems allow amount and type of materials used in experiments to be recorded. This paper focuses on providing the last piece: determining where the materials are deposited. Using a camera and projector setup over a lab bench, vision techniques allow a specially marked well tray and pipette to be located in real time with enough precision to determine which well the pipette tip is over. Using the projector, the tray can be augmented with relevant information, such as the next operation to be performed, or the contents of the tray. Without changing the biologist’s work practice, it is possible to record the physical interactions and provide easily available status and advice to the user. Preliminary user feedback suggests this system would indeed be a useful addition to the laboratory environment.

Implications for Location Systems in Indoor Wayfinding for Individuals with Cognitive Impairments

We studied an indoor wayfinding application for individuals with cognitive impairments using Wizard-of-Oz techniques. This gave us insights into the technological considerations for the location system underlying such an application. We have abstracted these into generally desirable properties for location-aware systems for wayfinding. In addition to general requirements for accuracy and robustness, we discuss what other information about the environment should be provided that supports wayfinding. Our work showed that combining rich environment information with user location is key to making indoor wayfinding applications feasible