Mathieu Brédif

TerraMobilita/iQmulus urban point cloud analysis benchmark

The objective of the TerraMobilita/iQmulus 3D urban analysis benchmark is to evaluate the current state of the art in urban scene analysis from mobile laser scanning (MLS) at large scale. A very detailed semantic tree for urban scenes is proposed. We call analysis the capacity of a method to separate the points of the scene into these categories (classification), and to separate the different objects of the same type for object classes (detection). A very large ground truth is produced manually in two steps using advanced editing tools developed especially for this benchmark. Based on this ground truth, the benchmark aims at evaluating the classification, detection and segmentation quality of the submitted results. Graphical abstractDisplay Omitted HighlightsVery rich data: high accuracy, high resolution, many attributes.Massive data: 160 million annotated points thanks to a performant web based annotation tool (and many hours of work).Rich semantics organized in a semantic tree with various levels of generalization.Very objective evaluation metrics.

Visibility estimation and joint inpainting of lidar depth maps

This paper presents a novel variational image inpainting method to solve the problem of generating, from 3-D lidar measures, a dense depth map coherent with a given color image, tackling visibility issues. When projecting the lidar point cloud onto the image plane, we generally obtain a sparse depth map, due to undersampling. Moreover, lidar and image sensor positions generally differ during acquisition, such that depth values referring to objects that are hidden from the image view point might appear with a naive projection. The proposed algorithm estimates the complete depth map, while simultaneously detecting and excluding those hidden points. It consists in a primal-dual optimization method, where a coupled total variation regularization term is included to match the depth and image gradients and a visibility indicator handles the selection of visible points. Tests with real data prove the effectiveness of the proposed strategy.

A web-based 3D mapping application using WebGL allowing interaction with images, point clouds and models

Nowadays we see a lot of Rich Internet Applications (RIA), but real interactive 3D applications have just started years ago, and it is only in 2011 that a GPU library was integrated directly in the major internet browsers. This opened a new way of creating 3D applications on the web and a new public, as much more users can access them without complicated plugin installations and this is just what 3D web mapping was waiting for. We present our street view application allowing to move through cities from a pedestrian point of view and interact with it in order to create and update precise urban maps enabling accessibility diagnostics, services and applications around mobility. We cared about precision in the data acquisition (images and laser) in order to be able to make accurate 3D measurements for professional usages. The user can show large laser clouds in his web-browser, access information on any points, draw 3D bounding boxes and export selections to databases. He can also make measurements from images, integrate collada models, display OpenData layers, etc. Finally we use projective texture mapping in real time to texture any mesh with our images.

Constrained palette-space exploration

Color palettes are widely used by artists to define colors of artworks and explore color designs. In general, artists select the colors of a palette by following a set of rules, e.g. contrast or relative luminance. Existing interactive palette exploration tools explore palette spaces following limited constraints defined as geometric configurations in color space e.g. harmony rules on the color wheel. Palette search algorithms sample palettes from color relations learned from an input dataset, however they cannot provide interactive user edits and palette refinement. We introduce in this work a new versatile formulation enabling the creation of constraint-based interactive palette exploration systems. Our technical contribution is a graph-based palette representation, from which we define palette exploration as a minimization problem that can be solved efficiently and provide real-time feedback. Based on our formulation, we introduce two interactive palette exploration strategies: constrained palette exploration, and for the first time, constrained palette interpolation. We demonstrate the performances of our approach on various application cases and evaluate how it helps users finding trade-offs between concurrent constraints.

A Stochastic Method for the Generation of Optimized Building Layouts Respecting Urban Regulations

The development in an urban area normally has to obey planning regulations. In France, such regulations are specified in local urban planning schemes (LUPS or PLU in French) defining the right to build at municipal or inter-municipal level. Many aspects are concerned in a PLU. We address to the spatial aspect defining the rules for building development. Since these rules are stated in technical documents, it’s not easy to comprehend or to assess their impacts. Driven by such issues, we propose to generate 3D building layouts that comply with the rules and have optimized indicators (e.g. floor area ratio), which is optional but useful. A building layout is a configuration of a number of buildings with various shapes (simplified as 3D boxes in this work). Thus, it can be seen as a realization of a marked point process (MPP) of 3D boxes, whose probability distribution can be defined through Gibbs energy with regard to a reference process. Its energy component reflects the compliance with the PLU rules in our problem. By maximizing this probability the optimal building layout can be found. The optimization task is realized by trans-dimensional simulated annealing (TDSA) coupled with a Reversible Jump Markov Chain Monte Carlo (RJMCMC) sampler. Several common types of the French PLU rules are studied and modeled into energy terms, and a case study is conducted to validate our approach.

3D roof details by 3D aerial vision

Virtual cities in a 3D GIS will not be complete without the details of roofs with chimneys, dormer windows, skylights. We contribute a method of extracting such details from digital large format aerial photography. The dense point clouds produced by a photogrammetric process produce major and minor plane surfaces of a roof and enter into a classification of its superstructures. In real city data with a total of 1312 superstructures the proposed automated process finds 1024, thus achieving success at a rate of 78%.

Progressive streaming and massive rendering of 3D city models on web-based virtual globe

The need for the real-time interactive co-visualization of 3D urban environments on a Web-based virtual Globe arises naturally in GIS but it still remains challenging due to the complexity of city models and their huge data sizes which largely overload the computational power and memory capacity of client devices. Especially on the Web, the visualization of city models makes their rendering not real-time because of the lack of content adaptation and progressive data transmission. This paper presents technical solutions for the co-visualization of massive city models in a Web-based virtual globe, allowing navigation over 3D cities on the globe in real-time. The volume of 3D city data, such as building data, does not allow us to render them directly, nor to keep them in the main memory. We propose to use not only a hierarchical presentation of geo-spatial data to create a chunk-based multiple resolution data structure which reduces complexity of the geometry being rendered; but also a view dependent algorithm so that only small subsets of 3D city models are streamed progressively in real-time and kept in client memory to contribute efficiently to the rendered image. Experimental results show that we can navigate over 3D cities on the Globe in real-time.

Planar roof surface segmentation using 3D vision

Internet search has initially been a strong driving force for the rapid emergence of 3D building models of large urban areas. Additionally, many commercial and governmental initiatives have been started to develop urban 3D geographic information systems in a transition from the classical 2D- to the novel 3D-GIS. The modeling of building roofs is thus a relevant research topic. The focus has been on the use of aerial LiDAR point clouds (Light Detection And Ranging). However, recent progress in digital aerial cameras has rendered possible the acquisition of very dense point clouds from high overlap digital aerial imagery, and to use these point clouds jointly with the image information to generate 3D building models. This paper presents a multi-step processing framework and work flow for the automatic segmentation of building roofs in densely built-up areas from high-resolution vertical aerial images. Details extruding from, or intruding into, a roof are being excluded so that each roof is being modeled by means of its planar segments and can then be classified as a specific roof type from a set of standard roof shapes. Our experimental work employs a test area in Graz (Austria) with 186 buildings.

Landmark based localization: LBA refinement using MCMC-optimized projections of RJMCMC-extracted road marks

Precise localization in dense urban areas is a challenging task for both mobile mapping and driver assistance systems. This paper proposes a strategy to use road markings as localization landmarks for vision based systems. First step consists in reconstructing a map of road marks. A mobile mapping system equipped with precise georeferencing devices is applied to scan the scene in 3D and to generate an ortho-image of the road surface. A RJMCMC sampler that is coupled with a simulated annealing method is applied to detect occurrences of road marking templates instanced from an extensible database of road mark patterns. The detected objects are reconstructed in 3D using the height information obtained from 3D points. A calibrated camera and a low cost GPS receiver are embedded on a vehicle and used as localization devices. Local bundle adjustment (LBA) is applied to estimate the trajectory of the vehicle. In order to reduce the drift of the trajectory, images are matched with the reconstructed road marks frequently. The matching is initialized by the initial poses that are estimated by LBA and optimized by a MCMC algorithm. The matching provides ground control points that are integrated in the LBA in order to refine the pose parameters. The method is evaluated on a set of images acquired in a real urban area and is compared with a precise ground-truth.

3D heterogeneous interactive web mapping application

The internet browsers nowadays show incredible possibilities with HTML5. It makes it possible to use all the power of your device such as the GPU and all its sensors, GPS, accelerometer, camera, etc. The ability to put hardware-accelerated 3D content in the browser provides a way for the creation of new web based applications that were previously the exclusive domain of the desktop environment. This paper introduces a novel implementation of a 3D GIS WebGL-based navigation system which allows end-users to navigate in a 3D realistic and immersive urban scene, to interact with different spatial data such as panoramic image, laser, 3D-city model, and vector data with modern functionalities such as using your smartphone as a remote, render for 3D screen and make the scene dynamic.