Patrick Morris Luley

Visual Object Detection for Mobile Road Sign Inventory

For road sign inventory and maintenance, we propose to use a mobile system based on a handheld device, GPS sensor, a camera, and a standard mobile GIS software. Camera images are then analysed via object recognition algorithms which results in an automated detection, i.e., localisation and classification of the signs. We present here the localisation of points and regions of interest, the fitting of geometrical constraints to the extracted set of interest points, and the matching of content information from the visual information within the sign plate. From the preliminary operational state of the vision based road sign detection system we conclude that the selected methodology is efficient enough to achieve the requested high quality in object detection and classification.

Smartphone eye tracking toolbox: accurate gaze recovery on mobile displays

Human Interaction with mobile devices has recently been estab-lished as application field in eye tracking research. Current technologies for gaze recovery on mobile displays cannot enable fully natural interaction with the mobile device: users are condi-tioned to interact with tightly mounted displays or distracted by markers in their view. We propose a novel approach that cap-tures point-of-regards (PORs) with eye tracking glasses (ETG) and then uses computer vision methodology for the robust local-ization of the smartphone in the head camera video. We present an integrated software package, i.e., the Smartphone Eye Track-ing Toolbox (SMET) that enables accurate gaze recovery on mobile displays with heat mapping of recent attention. We re-port the performance of the computer vision approach and demonstrate it with various natural interaction scenarios using the SMET Toolbox, enable ROI settings on the mobile display and show results from eye movement analysis, such as, ROI dwell time and statistics on eye gaze event (saccades, fixations).

Multisensor data fusion for high accuracy positioning on mobile phones

Analysis of human geographical orientation is a crucial issue to understand the users actual demand on context based information. As the limitations in accuracy of satellite based positioning especially in urban environments and network based positioning are well known, a novel framework concept based on data fusion of multiple sensors build-in handsets will enable to characterize the users situation and allow automated analysis with semantic mapping functionality. In this paper, the first approach for high accuracy multi sensor orientation tracking by the use of a mobile phone is discussed.

A tourism information system for rural areas based on a multi platform concept

Tourism information is predominantly based on geographically related information and therefore, the tourism and leisure industries are currently searching for ways how to explore the potential of technologies for presenting geographical data. In this paper a concept and its realization for a multi-platform solution for a geo-multimedia tourism information system are briefly described. In general the system is targeting on two different user groups, the tourism boards as service providers and the tourists as end-users. The concept covers an efficient data management for the service providers and state of the art visualization techniques for online, offline and mobile solutions. These cover the 2D as well as the 3D visualization of geo related tourism information and also interfaces to third party platforms. Such platforms like Google Earth and Google Maps became even more important in the last year due to their extensive dissemination.

A Multi-Sensor System for Mobile Services with Vision Enhanced Object and Location Awareness

We describe a system which proposes a solution for multi-sensor object awareness and positioning to enable stable location awareness for a mobile service in urban areas. The system offers technology of outdoors vision based object recognition that will extend state-of-the-art location and context aware services towards object based awareness in urban environments. In the proposed application scenario, tourist pedestrians are equipped with a GPRS or UMTS capable camera-phone. They are interested whether their field of view contains tourist sights that would point to more detailed information. Multimedia type data about related history might be explored by a mobile user who is intending to learn within the urban environment. Ambient learning is in this way achieved by pointing the device towards the urban sight, capturing an image, and consequently getting information about the object on site and within the focus of attention, i.e., the users current field of view. The described mobile system offers multiple opportunities for application in both mobile business and commerce, and is currently developed towards an industrial prototype

Visual object detection from mobile phone imagery for context awareness

We describe a system which proposes a solution for multi-sensor object awareness and positioning to enable stable location awareness for a mobile service in urban areas. The system offers technology of outdoors vision based object recognition that will extend state-of-the-art location and context aware services towards object based awareness in urban environments. In the proposed application scenario, tourist pedestrians are equipped with a GPRS or UMTS capable camera-phone. They are interested whether their field of view contains tourist sights that would point to more detailed information. Multimedia type data about related history might be explored by a mobile user who is intending to learn within the urban environment. Ambient learning is in this way achieved by pointing the device towards an urban sight, capturing an image, and consequently getting information about the object on site and within the focus of attention, i.e., the users current field of view. The described mobile system offers multiple opportunities for application in both mobile business and commerce, and is currently developed as an industrial prototype.

Mobile Augmented Reality for Tourists - MARFT

The aim of the project MARFT is to demonstrate the next generation of augmented reality targeting current mass market mobile phones. MARFT sets out to launch an interactive service for tourists visiting mountainous rural regions. During local trips they will be able to explore the surrounding landscape by pointing the lens of the smart-phone camera towards the area of interest. As soon as the view-finder shows the area of interest, the tourist will be able to choose between two products: (i) an augmented photo superimposed with tourist information like hiking tours or lookout points or (ii) a rendered 3D virtual reality view showing the same view as the real photo also augmented with tourist objects. The outstanding step beyond current augmented reality applications is that MARFT is able to augment the reality with cartographic accuracy. In addition to the benefit of presenting reliable information, MARFT is able to consider the visibility of objects and further to work completely offline in order to avoid roaming costs especially for tourists visiting from abroad.

Geo-contextual priors for attentive urban object recognition

Mobile vision services have recently been proposed for the support of urban nomadic users. While camera phones with image based recognition of urban objects provide intuitive interfaces for the exploration of urban space and mobile work, similar methodology can be applied to vision in mobile robots and autonomous aerial vehicles. A major issue for the performance of the service - involving indexing into a huge amount of reference images - is ambiguity in the visual information. We propose to exploit geo-information in association with visual features to restrict the search within a local context. In a mobile image retrieval task of urban object recognition, we determine object hypotheses from (i) mobile image based appearance and (ii) GPS based positioning, and investigate the performance of Bayesian information fusion with respect to benchmark geo-referenced image databases (TSG-20, TSG-40). This work specifically proposes to introduce position information as geo-contextual priors for geo-attention based object recognition to better prime the vision task. The results from geo-referenced image capture in an urban scenario prove a significant increase in recognition accuracy (≫ 10%) when using the geo-contextual information in contrast to omitting geo-information, the application of geo-attention is capable to improve accuracy by further ≫ 5%.

Geo-Services and Computer Vision for Object Awareness in Mobile System Applications

In recent years, location and context aware systems have been presented for the indexing and annotation of both location and user state relevant information to the user. These systems were mostly based on georeferencing from GPS signals, and driven by changes of the system state in dependence on situations that impact the overall system performance (user position, energy consumption, responsiveness, etc.). In this chapter we focus attention on a completely innovative aspect of contextual indexing for mobile system applications. We claim that the application of geo-services and vision based sensing enables systems to determine object characteristics information that can be used for semantic indexing of information and thereby provide an innovative quality of service.

Challenges in context-aware mobile language learning: the MASELTOV approach

Smartphones, as highly portable networked computing devices with embedded sensors including GPS receivers, are ideal platforms to support context-aware language learning. They can enable learning when the user is en-gaged in everyday activities while out and about, complementing formal language classes. A significant challenge, however, has been the practical implementation of services that can accurately identify and make use of context, particularly location, to offer meaningful language learning recommendations to users. In this paper we review a range of approaches to identifying context to support mobile language learning. We consider how dynamically changing aspects of context may influence the quality of recommendations presented to a user. We introduce the MASELTOV project’s use of context awareness combined with a rules-based recommendation engine to present suitable learning content to recent immigrants in urban areas; a group that may benefit from contextual support and can use the city as a learning environment.