Myriam Servières

A new typology of augmented reality applications

In recent years Augmented Reality (AR) has become more and more popular, especially since the availability of mobile devices, such as smartphones or tablets, brought AR into our everyday life. Although the AR community has not yet agreed on a formal definition of AR, some work focused on proposing classifications of existing AR methods or applications. Such applications cover a wide variety of technologies, devices and goals, consequently existing taxonomies rely on multiple classification criteria that try to take into account AR applications diversity. In this paper we review existing taxonomies of augmented reality applications and we propose our own, which is based on (1) the number of degrees of freedom required by the tracking of the application, as well as on (2) the visualization mode used, (3) the temporal base of the displayed content and (4) the rendering modalities used in the application. Our taxonomy covers location-based services as well as more traditional vision-based AR applications. Although AR is mainly based on the visual sense, other rendering modalities are also covered by the same degree-of-freedom criterion in our classification.

Camera tracking by online learning of keypoint arrangements using LLAH in augmented reality applications

We propose a camera-tracking method by on-line learning of keypoint arrangements in augmented reality applications. As target objects, we deal with intersection maps from GIS and text documents, which are not dealt with by the popular SIFT and SURF descriptors. For keypoint matching by keypoint arrangement, we use locally likely arrangement hashing (LLAH), in which the descriptors of the arrangement in a viewpoint are not invariant to the wide range of viewpoints because the arrangement is changeable with respect to viewpoints. In order to solve this problem, we propose online learning of descriptors using new configurations of keypoints at new viewpoints. The proposed method allows keypoint matching to proceed under new viewpoints. We evaluate the performance and robustness of our tracking method using view changes.

AR City Representation System Based on Map Recognition Using Topological Information

This paper presents a system for overlaying 3D GIS data information such as 3D buildings onto a 2D physical urban map. We propose a map recognition framework by analysis of distribution of local intersections in order to recognize the area of the physical map from a whole map. The retrieval of the geographical area described by the physical map is based on a hashing scheme, which is called LLAH. In the results, we will show some applications augmenting additional information on the map.

Validation of Mojette reconstruction from Radon acquisitions

Two new methods to perform interpolation mapping from Radon sinogram to Mojette domain are presented. Reconstructions are made from both spaces using FBP and SART algorithms. Assessment of the methods is made both from Shepp-Logan phantom and actual data and demonstrate the efficiency of the proposed algorithms.

Towards outdoor localization from GIS data and 3D content extracted from videos

We present an outdoor localization approach from videos and a 2D GIS (Geographic Information System). First, a 3D reconstruction as a cloud of points is done from the video with a Structure From Motion approach. Then a set of vertical planes is extracted from the cloud of points, using the presence of vanishing points in the video frames. This leads to a rough 3D model of the scene as a set of neighbor facades. That reconstruction will serve as an input data in a GIS localization query. Results of our reconstruction method are shown, based on a set of 27 videos of urban scenes.

Video/GIS registration system based on skyline matching method

For applications such as outdoor Augmented Reality (AR) or 3D City Model construction of updating, a real-time registration between a video sequence and a Geographic Information System (GIS) is required. In this work, we present a registration system using a GPS prior and a skyline matching method to estimate the camera pose from images. A skyline rectification method through vertical vanishing point detection is also developed to enable arbitrary camera orientation, with non null tilt. The proposed approach is robust, fully automatic and computationally inexpensive which makes it a possible solution for mobile device applications. The performance of our approach is demonstrated in several experimental evaluations. The possible refinement of camera location through a systematic method is also investigated.

Hybrid visual and inertial position and orientation estimation based on known urban 3D models

More and more pedestrians own devices (as a smartphone) that integrate a wide array of low-cost sensors (camera, IMU, magnetometer and GNSS receiver). GNSS is usually used for pedestrian localization in urban environment, but signal suffers of an inaccuracy of several meters. In order to have a more accurate localization and improve pedestrian navigation and urban mobility, we present a method for city-scale localization with a handheld device. Our central idea is to estimate the 3D location and 3D orientation of the phone camera based on the knowledge of the street furnitures, which have a high repeatability and a large coverage area in the city. Firstly, the use of inertial measurements acquired with an IMU in the vision based method allows to accelerate the calculation of the position and orientation. Secondly, the weighted fusion between the rotation matrices calculated with the vision and the inertial processes allows to give the more importance in the calculation with the highest confidence. With a good points selection, this provides a localization that is in the GNSS post-processed measurement precision use for determining the position and the orientation of the street furnitures. Performances are presented in terms of accuracy of positionning. The final aim is to have with our method a precision good enough to be able to propose in future works a on site display in augmented reality.

Design of a visual query language for geographic information system on a touch screen

This paper presents two spatial query methods for a Geographic Information System (GIS) that runs on a touch screen. On conventional GIS interfaces SQL is used to construct spatial queries. However keyboard typing proves to be inefficient on touch screens. Furthermore, SQL is not an easy-learning language, especially for novices to GIS. To simplify query construction, firstly we have designed a map interaction based query method (MIBQM). This method allows users to make simple queries by selecting necessary layers, features and query operators directly on the interface. To allow users to construct complex queries, a sketch drawing based query method (SDBQM) is proposed. Spatial query concepts can be represented by sketches of some symbolic graphical objects. It is possible to add spatial conditions and non-spatial conditions to describe query concepts more precisely. An evaluation has been made to compare SQL and MIBQM. We have found that for simple queries, MIBQM takes less time and proves to be more user-friendly.

Transforming conceptual spatiotemporal model into object model with semantic keeping

Our work is developed in the context of spatiotemporal data modeling and more particularly of urban data modeling. Our goal is to propose a methodology describing the overall process of urban data modeling from the design phase to the implementation in an information system. For that, we propose, within a GIS, an approach based on a conceptual model specific to spatiotemporal data modeling MADS, and on an ODBMS for storing and manipulating data. MADS uses spatial and temporal types that are more precise than those of programming languages on which are based ODBMS. MADS is semantically richer than the object model, and the purpose of this paper is to present the transformation rules of this conceptual model into an object model that keeps (as close as possible) its semantics.

Intelligent Reverse-Engineering Segmentation: Automatic Semantic Recognition of Large 3D Digitalized Cloud of Points Dedicated to Heritage Objects

In this article we present a multidisciplinary experimentation realized between a mechanical laboratory, a computer scientist laboratory and a museum. Our goal is to provide automatic tools for non-expert people who want to use 3D digitized elements. After scanning an objet, we obtain a huge amount of points. In order to manipulate it, it is necessary to decimate it. However, when doing this operation, we can optimize the algorithms for creating semantic topology; obviously we can do it automatically. Consequently, we are going to do what we name segmentation: we extract meaning from 3D points and meshes. Our experimentation deals with a physical mock-up of Nantes city that have been designed in 1900. After digitalization, we have created a software that can: 1. use the whole 3D cloud of points as an input; 2. fill a knowledge database with an intelligent segmentation of the 3D virtual models: ground, walls, roofs... This use case is the first step of our research. At the end, we aim to deploy our method to complex mechanical parts. Nowadays, when designing CAD parts we use as well as volume parts than surface parts or meshes. We know is it not necessary to reconstruct all the triangles. It is a lost of time and we can directly use cloud of points for CAD design. However, the design tree will not be updated. So, with our method, imagine that one day we can digitalize a motor and a system could automatically create the 3D mock-up and the design tree.