Automatic reconstruction of fully volumetric 3D building models from point clouds
PPreprint / ISPRS Journal of Photogrammetry and Remote Sensing (2019)
Automatic reconstruction of fully volumetric 3D building models from point clouds
Sebastian Ochmann a, ∗ , Richard Vock a , Reinhard Klein a a University of Bonn, Institute of Computer Science II, Endenicher Allee 19a, 53115 Bonn, Germany
A R T I C L E I N F O
Article history :Received July 2, 2019
Keywords:
Indoor Building Reconstruc-tion, Point Cloud Processing, IntegerLinear Programming, Building Informa-tion Modeling
A B S T R A C TWe present a novel method for reconstructing parametric, volumetric, multi-story build-ing models from unstructured, unfiltered indoor point clouds by means of solving aninteger linear optimization problem. Our approach overcomes limitations of previousmethods in several ways: First, we drop assumptions about the input data such as theavailability of separate scans as an initial room segmentation. Instead, a fully automaticroom segmentation and outlier removal is performed on the unstructured point clouds.Second, restricting the solution space of our optimization approach to arrangementsof volumetric wall entities representing the structure of a building enforces a consis-tent model of volumetric, interconnected walls fitted to the observed data instead ofunconnected, paper-thin surfaces. Third, we formulate the optimization as an integerlinear programming problem which allows for an exact solution instead of the approx-imations achieved with most previous techniques. Lastly, our optimization approach isdesigned to incorporate hard constraints which were di ffi cult or even impossible to in-tegrate before. We evaluate and demonstrate the capabilities of our proposed approachon a variety of complex real-world point clouds. This is an early preprint version. The final version, which wasaccepted for publication in the ISPRS Journal of Photogram-metry and Remote Sensing, is available at https: // doi. org/ 10. 1016/ j. isprsjprs. 2019. 03. 017
1. Introduction
The challenging problem of generating high-quality, three-dimensional building models from point cloud scans has beenapproached in a variety of ways in recent years by the com-puter graphics and remote sensing communities as well as inthe architecture domain. Especially for various applications inComputer Aided Design (CAD) and emerging fields such asBuilding Information Modeling (BIM), the reconstructed mod-els are usually required to adhere to industry-standard specifi-cations such as the Industry Foundation Classes (IFC). In con-trast to the representation of a building in the form of e.g. anunordered point cloud, a set of unconnected surfaces, or bound-ary meshes, a BIM / IFC model closely resembles the physicalbuilding structure by defining buildings as semantically anno-tated, volumetric building entities such as walls and floor slabs,usually including additional information regarding how theseelements are interconnected. ∗ Corresponding author: e-mail: [email protected]
Most previous approaches focus on the reconstruction ofcompletely separate, planar surfaces without additional infor-mation regarding how they relate to each other [1], or on rep-resenting buildings as watertight boundaries of either the wholebuilding [2] or separate rooms [3, 4] and are thereby lackingin providing insights into the building structure. Also, assump-tions such as the one that stories can be globally separated byhorizontal planes are very limiting in practice. None of theseapproaches yields a representation which enables unhinderedusage in the aforementioned scenario. While one recent ap-proach [5] does model the measured point cloud data using vol-umetric building entities, the method is restricted to single-storybuildings which limits its usability without laborious manualseparation of the point cloud data into separate stories. Ad-ditionally, generation of resulting wall and floor slab elementsis done in a post-processing step without being integrated intothe used optimization framework which may result in locallyimplausible results. Other methods [6, 7] aiming at reconstruct-ing true BIM models make the severe assumption that walls arepositioned in a Manhattan world constellation which is oftenviolated by real-world buildings.Our proposed method overcomes limitations of previous ap-proaches by alleviating the requirements on the input dataand by providing a flexible optimization framework for indoorbuilding reconstruction. Some prior methods (e.g. [4, 5]) re-quire separate scans and scan positions to derive an initial, a r X i v : . [ c s . G R ] J u l Preprint / ISPRS Journal of Photogrammetry and Remote Sensing (2019) coarse segmentation into rooms. In contrast, our fully auto-matic room segmentation approach, by design, does not dependon the availability of such information and does not impose par-ticular rules for scanning (e.g. one scan per room). Further-more, our novel integer linear programming formulation for thereconstruction problem provides flexible means to steer the re-construction process while globally constraining the solutionspace to feasible solutions, thus guaranteeing a plausible model.Additional information such as manually augmented hints mayoptionally be incorporated by means of hard constraints in orderto further guide the reconstruction process. While some previ-ous approaches regularize the resulting model based on roomboundary complexity, they fail to account for dependencies be-tween surfaces related by volumetric wall elements, e.g. oppos-ing surfaces between neighboring rooms. Our formulation ofthe solution space based on volumetric entities enables betterregularization of the model with respect to the actual volumet-ric walls and slabs used to represent the building. In contrastto any previous approach, the result of our optimization imme-diately yields the complete geometry of all reconstructed wallsand slabs as well as their volumetric intersections which allowsfor a direct generation of plausible BIM / IFC models.In summary, the main features of our approach are:1. Fully automatic, volumetric reconstruction including vol-umetric intersections between elements.2. Flexible integration of constraints to enforce global andlocal properties of the resulting model.Our main technical contributions are:1. Automatic filtering of outliers and room segmentation ofunstructured, multi-story 3D point clouds.2. A new formulation of the indoor reconstruction task as alinear integer programming problem that can be e ffi cientlysolved using o ff -the-shelf software.
2. Related Work
Research on scan-to-BIM and related approaches led to awide range of developments in recent years and still is a cur-rent topic of ongoing work. We first provide a comprehensiveoverview of methods dealing specifically with indoor buildingreconstruction which we then complement with a summary ofmore loosely related but complementary abstraction approachesand applications.
The works presented in this section are closely related to ourgoal of indoor building reconstruction. Table 1 summarizes andcompares key features of di ff erent approaches.Some methods aim at the generation of 2D floor plans. Okornet al. [16] model 2D floor plans by projecting detected struc-tures into the horizontal plane and performing wall segment de-tection based on the Hough transform. Ambrus¸ et al. [13] re-construct floor plans including a room labeling obtained usingan energy minimization approach. A deep neural architecturefor automatic floor plan generation from RGBD video has been presented by Liu et al. [7]. Using pixel-wise predictions of floorplan geometry and semantics, integer programming [17] is usedto recover a vector graphics reconstruction.Some approaches perform a reconstruction of individualrooms. Budroni et al. [8] reconstruct closed boundary repre-sentations of single rooms using plane sweep surface detectionand a 2D line arrangement with a split-and-merge approach.The methods by Ad´an et al. [9] and Xiong et al. [10] focus onrecovering detailed surface labelings, explicitly reasoning aboutocclusions using a ray-tracing approach. In a similar spirit, Pre-vitali et al. [12] perform a reconstruction of single rooms aspolyhedral models including ray-tracing based reasoning aboutocclusions and opening detection.Certain methods aim at the reconstruction of the building asa whole without explicitly considering room topology or seg-mentation. Sanchez et al. [1] represent buildings as polygonalsurface models including detection of smaller-scale structuressuch as parametric staircases. Oesau et al. [18] use a 2D celldecomposition to perform binary inside / outside labeling using aGraph-Cut based optimization. The detail level of this approachis enhanced by Oesau et al. in [2] by means of an improved linedetection strategy. With a similar goal of providing simplifiedenvironment maps for e.g. navigation, Xiao et al. [19] employconstructive solid geometry (CSG) operations to generate a vol-umetric wall model. Room topology is not explicitly modeled.Many recent methods approach the reconstruction problemin a 2.5D setting, including a segmentation into separate rooms.Mura et al. [20, 11] model buildings as 2.5D polyhedral meshesby means of constructing a 2D line arrangement and performing k -medoid clustering based on di ff usion embeddings. Mura et al.[21] also propose a related approach which allows arbitrary wallorientations and performs recursive clustering on a constrainedDelaunay tetrahedralization. The method by Turner et al. [22]provides e ffi cient means to generate 2.5D, textured meshes fore.g. navigation purposes including a room segmentation ob-tained by Graph-Cut in a triangulated environment map. Anextension providing enhanced texture mapping has been pre-sented in [3]. The reconstruction method by Wang et al. [15]models outer and inner walls by means of 2D line arrangementslabeled using di ff usion embeddings similar to [11]. They alsoreconstruct doors using a simulated ray casting approach. Mu-rali et al. [6] present a system to quickly generate BIM modelsfrom mobile devices such as Google Project Tango, MicrosoftKinect or Microsoft HoloLens, including semantic annotationsand relations between reconstructed elements. The approach iscurrently limited to single-story, Manhattan world buildings.Few approaches consider the more general case of slantedwalls or ceilings. Mura et al. [4] reconstruct polyhedral roomboundaries with arbitrary wall and ceiling orientations. Earlyrule-based classification of detected elements helps pruning in-valid parts. Room segmentation is performed by clusteringsteered by visible surface overlap. Mura et al. [23] proposean extension using automatically clustered synthetic viewpointsand show applicability on complex multi-story buildings.None of the aforementioned methods reconstruct volumetricwall and slab elements which are directly usable in a BIM set-ting. Few methods have approached this problem before. Stam- reprint / ISPRS Journal of Photogrammetry and Remote Sensing (2019) 3No scan positions Non-Manhattan Multiple rooms Full 3D recons. Slanted ceilings Volumetric wallsBudroni ’10 [8] (cid:51) (cid:55) (cid:55) (cid:55) (cid:55) (cid:55)
Ad´an ’11 [9] (cid:55) (cid:51) (cid:51) (cid:55) (cid:55) (cid:55)
Xiong ’13 [10] (cid:55) (cid:51) (cid:51) (cid:55) (cid:55) (cid:55)
Mura ’14 [11] (cid:55) (cid:51) (cid:51) (cid:55) (cid:55) (cid:55)
Oesau ’14 [2] (cid:51) (cid:51) (cid:55) (cid:55) (cid:55) (cid:55)
Previtali ’14 [12] (cid:55) (cid:51) (cid:55) (cid:55) (cid:55) (cid:55)
Turner ’15 [3] (cid:55) (cid:51) (cid:51) (cid:55) (cid:55) (cid:55)
Mura ’16 [4] (cid:55) (cid:51) (cid:51) (cid:51) (cid:51) (cid:55)
Ochmann ’16 [5] (cid:55) (cid:51) (cid:51) (cid:55) (cid:55) (cid:51)
Ambrus¸ ’17 [13] (cid:51) (cid:51) (cid:51) (cid:55) (cid:55) (cid:55) Macher ’17 [14] (cid:51) (cid:51) (cid:51) (cid:55) (cid:55) (cid:51) Murali ’17 [6] (cid:51) (cid:55) (cid:51) (cid:55) (cid:55) (cid:51)
Wang ’17 [15] (cid:55) (cid:51) (cid:51) (cid:55) (cid:55) (cid:55) Ours (cid:51) (cid:51) (cid:51) (cid:51) (cid:55) (cid:51)
Table 1. Feature comparison of recent indoor reconstruction approaches. Notes: Full 3D reconstruction means that arbitrary vertical room constellationsincluding rooms spanning multiple floors are handled correctly. Virtual scan positions are automatically estimated. Volumetric walls are generatedmanually in post-processing. Scanner trajectories are used. bler et al. [24] aim to generate volumetric 3D building modelsusing learning approaches for the classification and scoring ofdetected elements, and simulated annealing for optimizing theoverall model. The approach makes strong assumptions aboutthe input data, requiring both interior and exterior scans, as wellas scanner positions. Thomson et al. [25] generate volumetricwalls from point clouds by detecting planes using a RANSACapproach and fitting suitable IFC wall entities to the detectedsurfaces; room volumes and topology are not explicitly mod-eled. They also propose a point cloud segmentation schemebased on a corresponding IFC model. A method which explic-itly represents buildings as interconnected volumetric wall el-ements has been presented by Ochmann et al. [5]. They con-struct a 2D line arrangement of wall center lines representingpairs of opposing wall surfaces and perform a room labeling ofthe arrangement faces by means of a Graph-Cut based multi-label energy minimization. Multi-story buildings are not sup-ported. Macher et al. [14] propose a semi-automatic reconstruc-tion approach by first segmenting the input data automaticallyand exporting the result in an interim OBJ format, and subse-quently constructing an IFC file with manual intervention in apost-processing step.To our knowledge, our approach is the first to combine gen-eral multi-story, multi-room reconstruction with fully volumet-ric room and wall entities.
We now highlight some loosely related approaches whichpursue more general or complementary goals which may bebeneficial for tackling the reconstruction problem on di ff erentlevels. Monszpart et al. [26] represent man-made scenes (e.g.buildings) by a regular arrangement of planes, taking into ac-count non-local inter-primitive symmetry relations. Such a reg-ularization may be useful for various arrangement-based recon-struction approaches. A method for reconstructing lightweight,manifold, polygonal boundary models from point clouds hasbeen presented by Nan et al. [27]. They employ an in-side / outside labeling approach using binary linear program-ming. Jung et al. [28] generate watertight floor maps by meansof skeletonization in a 2D binary occupancy map with subse- quent labeling of separate rooms. A 3D room partitioning ap-proach using anisotropic potential fields with subsequent unsu-pervised clustering has been presented by Bobkov et al. [29].Pursuing a similar goal, Ochmann et al. [30] perform a seg-mentation of indoor point clouds into separate rooms usinga visibility-based approach. Openings between neighboringrooms are detected to obtain a room connectivity graph. Bassieret al. [31] employ a machine learning approach to classify struc-tural elements such as walls, floors, ceilings, and beams in pointcloud data. The method by Liu et al. [17] generates topologi-cally and geometrically consistent floor plans from 2D rasterimages using an integer programming approach. While the ap-proach is designed to work on 2D data and assumes Manhattanworld geometry, the idea to enforce global properties of the re-sulting model using integer programming is related to our work.Focusing on non-structural elements relevant to BIM models,Ad´an et al. [32] present an approach for detecting various im-portant entities such as sockets, switches, signs, and safety-related items. While the method by Son et al. [33] does not ex-plicitly model a building’s room topology, they detect variousimportant volumetric elements such as walls, slabs, columnsand beams, also taking into account material properties and re-lations between elements. Automated scan-to-BIM methods facilitate a range of diverseapplications in di ff erent areas such as construction surveillance,facility management, or energy simulations. Garwood et al.[34] propose a framework for storing building geometry in aformat suitable for e.g. energy simulation and verification tasks,and highlight the importance of fast, automated methods forobtaining suitable models. Hyland et al. [35] propose the us-age of open standards and automatically derived BIM modelsfrom measurements for performing automated compliance con-trol by comparing the as-built and as-designed states of build-ings. In a similar spirit, O’Kee ff e et al. [36] have developed val-idation approaches for determining and analyzing di ff erencesbetween scans and BIM models. A prototypical approach hasbeen presented by Brodie et al. [37] who propose a cloud-basedplatform integrating tools for generating models from and val- Preprint / ISPRS Journal of Photogrammetry and Remote Sensing (2019) a b cd e f
Fig. 1. Overview of the main steps. Ceiling of upper floor is hidden in (a), (c), and (f) for visualization purposes. (a) The input is a registered but otherwiseunstructured and unfiltered indoor point cloud. (b) Planes are detected by means of RANSAC shape detection. (c) Outliers are automatically removedand rooms are segmented using an unsupervised clustering approach based on mutual visibility between point patches. (d) Detected planes are classifiedas horizontal slab surfaces and vertical wall surfaces (only latter shown). Surfaces are assigned multi-label support bitmaps. (e) A 3D plane arrangementis constructed by intersecting all planes, yielding a cell complex. Priors for rooms, outside area and surface support are estimated. (f) The final modelconsisting of interrelated room and wall volumes is obtained by solving a integer linear program in which cell labels are binary variables. idating models against point clouds. Krispel et al. [38] devel-oped a method for automatic detection of power sockets andfor the generation of hypotheses for electrical lines based onautomatically generated building models. An approach for in-tegrating IFC BIM models and point cloud data in a commonfile format has been presented by Krijnen et al. [39]. They high-light the semantically meaningful association of both worlds fordocumentation, structuring, annotation, synchronization and re-trieval tasks.
3. Overview
The input of our approach is a 3D indoor point cloud (Figure1 a) with oriented normals whose “up” direction is assumed tobe the z -axis. If normals are not yet available, they are estimatedby local Principal Component Analysis (PCA).We first detect planes using an e ffi cient RANSAC implemen-tation [40] (Figure 1 b) and compute occupancy bitmaps foreach detected plane from the respective supporting points.The detected planes are used to automatically eliminate out-lier points, and to determine point clusters corresponding toindividual rooms. This clustering is performed by means ofMarkov Clustering [41] which does not require prior informa-tion about the number of rooms and results in a labeling of thepoint cloud (Figure 1 c).The resulting point labels are projected to the previously de-tected planes and discretized into multi-label bitmaps. Planesare pruned, rectified, clustered, and classified as candidates forvertical wall or horizontal slab surfaces (Figure 1 d; only verti-cal surfaces shown for visualization purposes). Since we baseour reconstruction on volumetric walls and slabs instead of sin-gle surfaces, pairs of nearby, approximately parallel surfacesare grouped to wall and slab candidates. Based on promising previous approaches (e.g. [4, 23, 5, 13]),we then derive a three-dimensional arrangement of planes fromthe set of wall and slab candidates (Figure 1 e). To this end, allsurfaces are interpreted as infinite planes and intersected witheach other which results in a segmentation of 3D space intoconvex polyhedral cells. In particular, each wall and slab candi-date is represented by a set of cells located between the respec-tive two candidate surfaces. Priors for the existence of di ff erentrooms and wall surfaces are estimated for each 3D cell and 2Dface using the labeled surface candidates.The main step of our approach is to find a labeling of allcells such that each cell is either assigned to a room , or outsidespace . Additionally, volumetric walls must be placed wherevera transition between inside and outside space takes place whichis also modeled as part of the labeling problem. The labelingshould faithfully conform to the measured data and simultane-ously fulfill certain constraints (e.g. wall connectivity) to ensurea plausible resulting model (Figure 1 f).Formulating this task as an optimization problem requiresthree parts: First, we define a space of possible solutions withmeaningful priors to guide the solver. The geometry of thisspace is given by the arrangement of planes. Priors for loca-tions of rooms and walls in the cell complex are derived fromthe measured data. Second, we need to define constraints to re-strict the feasibility of a solution. They enforce that any solutionsatisfies predefined rules, e.g. a room and outside space must beseparated by a wall. Third, an objective function for assessingthe quality of a solution is formulated as a cost function whichis minimized under the given constraints.After a solution is found it can easily be converted into a for-mat suitable for rendering or exporting, e.g. an IFC file or amesh, by considering the cell labeling and the boundaries be-tween di ff erently labeled cells. reprint / ISPRS Journal of Photogrammetry and Remote Sensing (2019) 5
4. Method
In this Section, we provide details regarding each of the stepsinvolved in our approach with a focus on the formulation as anoptimization problem.
Based on the widely used assumption that the coarse geom-etry of most buildings can be represented (or su ffi ciently ap-proximated) by piecewise planar surfaces, a crucial first stepof our approach is the detection of planes in the point clouddata. To this end, an e ffi cient RANSAC approach [40] imple-mented in CGAL [42] is used. The most important parametersare maximum point-to-plane distance, normal angle threshold,minimum number of supporting points per plane, and the prob-ability to miss the largest plane candidate. These can usuallybe chosen depending on point cloud data quality and used for awide variety of datasets with similar characteristics (e.g. scan-ner type, density, noise level). The supporting points of eachplane are projected into occupancy bitmaps on the respectiveplane (Figure 2 b), yielding a discretized approximation of sup-port by measured points. Planes with low support area (esti-mated using the occupancy bitmaps) are pruned later (Section4.4). Since the relatively coarse occupancy bitmaps are inde-pendent of the point cloud density, the minimum number ofpoints for detecting a plane of the RANSAC algorithm may beset relatively low to cope with lower-resolution point clouds. Real-world point clouds often contain large amounts of out-lier points, often due to outside areas scanned through open-ings. In order to prune outlier points early in the process, weemploy a simple but very e ff ective ray casting approach sim-ilar to [5]. From each point p , n stochastically sampled rays r i , i = , . . . , n , are cast into the hemisphere oriented into thedirection of the normal at point p . Ray casting is performedagainst the occupancy bitmaps of the previously detected prim-itives. Let h ( r i ) be a hit function which is 1 if some surfacewas hit, and 0 otherwise. We approximate the probability that p lies inside of the building as in ( p ) = n (cid:80) ni = h ( r i ) . If in ( p ) isbelow a given threshold (in our experiments 0 . p is removedfrom the point cloud and the occupancy bitmaps of the planesare updated. This process is iterated a small number of times. Priors for the locations of rooms and outside area in three-dimensional space are vital for the later optimization step, evenif they are coarse estimations. We formulate the estimation ofpriors as a point cloud labeling problem where each label rep-resents either a room, or the outside area.Our proposed automatic labeling approach is based on theidea that regions of the point cloud with high mutual visibil-ity form clusters which correspond to rooms of the building.We implement this by performing visibility tests by means ofray casting between point patches on detected surfaces whichyields a visibility graph. Nodes of this graph are then clustered a bc d
Fig. 2. Detected planes are the basis for our reconstruction. Di ff erent kindsof bitmaps (i.e. grids) on the planes are used throughout the approach. (a)Unlabeled input point cloud. (b) Binary occupancy bitmaps on the detectedplanes (Section 4.1) are used as a lightweight representation of support bythe point cloud. Di ff erent planes are shown in di ff erent colors. (c) Raycasting against the occupancy bitmaps and clustering yields a segmentationof the point cloud into rooms (Section 4.3). Di ff erent room labels are shownin di ff erent colors. (d) The point cloud labeling is projected into multi-label bitmaps where each pixel contains a soft assignment [0 , n to the n di ff erent room labels as defined in Section 4.3. This is used for estimatinglocations of rooms in 3D space as described in Section 4.7. The multi-labelbitmaps are shown using the same label colors as in (c). In (b) and (d), onlyvertical planes are shown for clarity. by means of the Markov Clustering algorithm [41] which deter-mines natural clusters within the graph by flow simulation.Point patches are constructed by generating coarse occu-pancy bitmaps for each plane and considering each occupiedpixel as a patch with a normal identical to the respective planenormal. In our experiments, a patch size of 40cm × p i be the i -th patch with center posi-tion c i and normal n i . For each pair p i , p j , i (cid:44) j , ray castingbetween the points c i + ε n i and c j + ε n j , with ε : = p i , p j is set to 1, otherwise it is set to 0. This yieldsa visibility graph whose nodes are clustered using the MarkovClustering algorithm. The computed visibility is interpreted asflow between node pairs corresponding to the respective pointpatches. The main advantage of this method is that it is unsu-pervised and thus does not require a manual specification of thenumber of occurring labels.As a result, we obtain n disjoint clusters of patches whichbelong to di ff erent rooms and define the set of room labels R : = { r , . . . , r n } which will be used throughout the remainderof the reconstruction process. Each point of the point cloud isassigned the room label of the respective point patch. Note thatthe number of room labels n may be larger than the number ofrooms that will actually be contained in the final reconstruction. The detected planes usually include many surfaces which arenot part of walls, floors and ceilings. Even correctly detectedsurfaces will generally not be perfectly vertical or horizontal.
Preprint / ISPRS Journal of Photogrammetry and Remote Sensing (2019)
We thus apply a pruning, classification and rectification stepto extract two sets of candidates for wall and slab surfaces. Theoccupancy bitmaps are used to estimate the support area of eachsurface independently of point cloud density. Planes with sup-port below an area threshold as well as planes which are notapproximately vertical or horizontal are discarded. The remain-ing planes are classified as wall or slab surface candidates de-pending on their normal direction, and adjusted to be perfectlyhorizontal or vertical.As a prerequisite for later room prior estimation (Section4.7), each surface is also assigned a multi-label support bitmapwith continuous values in [0 ,
1] for each room label in R (Fig-ure 2 d). This provides a soft-assignment of di ff erent regionsof each surface to di ff erent room labels. The label bitmaps aregenerated by projecting all supporting points onto the respec-tive surface and averaging the previously determined point la-bels within each pixel.Furthermore, we dilate the support bitmaps. The rationaleis that reconstructed walls with no surface support by the pointcloud data are penalized by a cost function defined later in Sec-tion 4.8. Since we reconstruct wall intersections volumetrically,placing the respective wall entities in between rooms wouldcause high costs since surface support is naturally restricted toregions that are visible to the scanner (Figure 3, middle row).By slightly extending the surface support, we encourage con-struction of intersecting wall entities in regions with nearby sur-face support (Figure 3, bottom row). Since our approach is based on the notion of volumetric wallsand slabs instead of single surfaces, the next step is to determinepairs of opposing surfaces forming potential building elements.To this end, a simple pairing procedure is employed. For eachsurface, we search a matching, approximately parallel surfacewith opposing normal orientation within a user-defined distance
Fig. 3. Dilation of surface support. Top: Example point cloud viewed fromabove. Middle: Surface support (left, thick lines) is naturally restricted toparts visible to the scanner. This leads to high costs for the wall intersectionsince wall surfaces are not supported by points (right). Bottom: Dilatingthe surface support (left, dotted lines) extends support into the interiors ofwalls, encouraging placement of volumetric intersections (right). and angle threshold. If a match is found, the two surfaces arepaired to form a wall or slab candidate. It should be noted thata single surface may thus be part of multiple pairs. For sur-faces without any matching counterpart, virtual surfaces with auser-defined distance are added to the set of surfaces. This isusually the case for outside walls for which only the inner sidehas been scanned. This augmentation is important since inte-rior spaces are required to be bounded by volumetric walls orslabs. The m generated candidates constitute the set of wall / slablabels W : = { w , . . . , w m } . Which of these walls and slabs arecontained in the final model is decided by the optimization de-scribed in Section 4.8. The geometry of the search space for finding an optimal con-stellation of rooms, walls and slabs is modeled as an arrange-ment of planes and the
3D cell complex induced thereby. It isconstructed by intersecting all (infinite) planes of the wall andslab candidate surfaces with each other. Since vertical walls andhorizontal slabs are treated identically, we will hereafter refer toboth simply as walls .Cells of the arrangement are convex, three-dimensional sub-sets of the space inside and outside of the building. Each cellbelongs either to a room, or the outside area. Additionally, wallsmay be placed in cells that are part of the outside area. Con-straints such as that a cell may belong to at most one room,or that walls may only occur in the outside area (e.g. betweenrooms) are formulated as constraints for the optimization prob-lem in Section 4.8.Faces between neighboring cells are convex, two-dimensional subsets of regions on the planes of wall surfaces.Each face may separate di ff erent regions (e.g. a room and awall) from each other. For guiding the optimization, two kinds of priors are esti-mated from the data. First, volumetric priors for the existenceof di ff erent rooms as well as outside area are estimated for each3D cell of the arrangement. Second, support by the point clouddata is estimated for each 2D face between neighboring cells. Preparations.
The arrangement consists of cells C : = { c , . . . , c p } . For two cells, the notation c a (cid:97) c b means that c a , c b are neighboring and the normal of the separating oriented face f c a , c b points towards c a (Figure 4 a). The set of all oriented facesis denoted as F : = { f c a , c b | c a (cid:97) c b } . For brevity, we write f in-stead of f c a , c b if the specific incident cells are irrelevant.The set of room labels is R : = { r , . . . , r n } with n being thenumber of room clusters as introduced in Section 4.3, and theset of wall labels is W : = { w , . . . , w m } with m being the num-ber of generated wall candidates as introduced in Section 4.5.We furthermore define an additional outside label O : = { o } . Asdetailed later, the outside label is used for cells that are not theinterior space of a room. The union of rooms and outside labelsis denoted R o : = R ∪ O ; the set of all labels is L : = R ∪ O ∪ W .Let C w ⊆ C be the set of cells that are contained in wall can-didate w , i.e. all cells that are located between the two surfaces reprint / ISPRS Journal of Photogrammetry and Remote Sensing (2019) 7 a b cd e f
Fig. 4. Explanation of notation and constraints. (a) Neighboring cells are considered as ordered pairs c a (cid:97) c b with respect to normal orientation of theseparating face f c a , c b . (b) Considering a cell c , W c is the set of wall candidates enclosing cell c . A face f c a , c b may be a boundary or an inner face of a set ofwalls. The set of boundary walls with respect to that face is W c a c b , the set of inner walls is W c a c b . (c) Transitions between interior and exterior area at aface f may only occur with the room label being on the positive side of f (Constraint 2). (d) If a face f is the boundary of a room, some wall needs to beactive on the negative side of f (Constraint 4). (e) A wall may end at an inner face f only if the wall label is on the negative side of f (Constraint 5). (f) If awall w ends at an inner face, this face must be a boundary face of at least one other active wall w . This enforces connectedness between walls. of w . Conversely, W c : = { w ∈ W | c ∈ C w } is the set of wallsthat contain cell c .For a particular cell pair c a , c b , we define the set W c a c b ofwalls that are contained in cell c b but not in c a , i.e. W c a c b : = W c b \ W c a . The separating face f c a , c b is called a boundary face of the wallsin W c a c b . Analogously, we define the set W c a c b of walls thatare contained in both c a and c b , i.e. W c a c b : = W c a ∩ W c b . The separating face f c a , c b is called an inner face of the walls in W c a c b . These definitions are exemplified in Figure 4 b.It should be noted that an inner face of a wall w is always theboundary face of another wall (which is often approximatelyperpendicular to w ). As an example, in Figure 4 b, face f c a , c b isan inner face of wall w and a boundary face of the intersectingwall w . This will become important for the definition of theoptimization constraints in Section 4.8. Room and outside priors.
To estimate probabilities where dif-ferent rooms and outside area are located in 3D space, we es-timate a volumetric prior function p C ( c , l ) : C × R o → [0 , ff a label l is likely to occur withina cell c . To this end, we perform stochastic ray casting fromsampled points in 3D space and average previously computedroom labels on surfaces visible from each point. For each cell c , k random points are sampled within c . To draw enough sam-ples for narrow cells, which are very common due to parallelsurfaces, k is chosen proportional tomax( volume ( c ) , diameter ( c )) . Centered at each sampled point, d rays are cast into random di-rections. p C ( c , l i ) , i = , . . . , n , is then set to the average over allobserved room labels. Rays hitting the back side of surfaces, aswell as rays without surface intersections, are counted as out-side . Face support priors.
In addition to the volumetric room andoutside prior function, we estimate a face support function p F ( f ) : F → [0 ,
1] which returns a high value i ff a face f issupported by the point cloud. This function is later used for se-lecting probable wall candidates and regularizing the optimiza-tion result. To estimate p F ( f ) for a face f , we first sample k random points within f where k is proportional tomax( area ( f ) , diameter ( f )) . Subsequently, all sampled points are projected onto the surfacefrom which face f was generated in the arrangement. p F ( f ) isthen set to the ratio between the number of sampled points lyingwithin the support approximated by the occupancy bitmap ofthe respective surface to the total number of sampled points. For finding an optimal labeling of all cells, we employ a 0-1integer linear programming approach in which binary variablesfor each cell are interpreted as room, outside, and wall labelassignments to cells. This approach has the advantage that aset of rules to be fulfilled by any feasible solution can be for-mulated as hard constraints. Approximate multi-label methodsbased on e.g. Graph Cuts [43] are more restricted regarding thefamily of objective functions and constraints that can be usedand may fail to find good solutions if the objective is not su ffi -ciently smooth. We first discuss the set of constraints imposedon our model before defining the objective function. Preprint / ISPRS Journal of Photogrammetry and Remote Sensing (2019)
Preparations.
Each of the binary variables x c , l ∈ { , } , c ∈ C , l ∈ L , of our optimization is a binary assignment of a label l to a cell c .A value of 1 means that the label is assigned or active . It shouldbe noted that a cell is not necessarily assigned only a singlelabel. In particular, a cell can be assigned the outside label and a nonempty set of wall labels at the same time as defined by theconstraints below. Also, cells where walls intersect are assigned all labels of the intersecting walls. We also use the notion of inner and boundary faces as defined in Section 4.7. Constraint 1.
Each cell c must be assigned exactly one labelfrom R o , i.e. ∀ c ∈ C : (cid:88) r ∈R o x c , r = . (1) Constraint 2.
At boundary faces of room interiors, the roomlabel may only occur on the positive side of the separating face,i.e. ∀ f c a , c b ∈ F ∀ r ∈ R : x c a , r − x c b , r ≥ , (2)as shown in Figure 4 c. Note that this constraint implies thattwo di ff erent room labels r p (cid:44) r q cannot be directly neighboringsince this would violate the constraint for one of the room la-bels. As a consequence, this avoids “paper thin” walls betweenrooms since they must be separated by outside area, therebyfollowing the physical nature inherent to walls. Constraint 3.
Wall labels may only occur in cells which areassigned the outside label, i.e. ∀ c ∈ C ∀ w ∈ W c : x c , w ≤ x c , o . (3) Constraint 4.
The boundary faces of room interiors must alsobe the boundary faces of an active wall, i.e. ∀ f c a , c b ∈ F : (cid:88) w ∈W cacb x c b , w ≥ x c b , o − x c a , o , (4)as illustrated Figure 4 d. This constraint implies that there can-not be a transition between room interior and outside area with-out activating a wall at all faces where the transition occurs. Constraint 5.
At wall boundaries which occur at inner faces,the wall label must be on the negative side of the respectivefaces, i.e. ∀ f c a , c b ∈ F ∀ w ∈ W c a c b : x c b , w − x c a , w ≥ , (5)as exemplified Figure 4 e. This constraint is a prerequisite forConstraint 6 as well as the objective function which require theleft-hand side expression to be nonnegative. Constraint 6.
A wall may end at an inner face only if thisface is a boundary face of at least one other active wall, i.e. ∀ f c a , c b ∈ F ∀ w ∈ W c a c b : (cid:88) w (cid:48) ∈W cacb x c b , w (cid:48) ≥ x c b , w − x c a , w , (6)as depicted Figure 4 f. This constraint enforces that walls areinterconnected at their endpoints since it disallows that a wallends at an inner face without it coinciding with a boundary faceof an active wall. Objective function.
To determine the optimal labeling, we de-fine a cost function F C for a solution over the for cell complex C of the form F C : = − R C + α ( W F b + W F i ) , (7)consisting of the following terms. The volumetric room andoutside area fitness term R C rewards the assignment of the mostlikely labels for each cell c ∈ C and is defined as R C : = (cid:88) c ∈C (cid:88) r ∈R o x c , r · p C ( c , r ) · volume ( c ) , (8)where x c , r denotes the binary variable for the assignment of la-bel r to cell c and p C ( c , r ) represents the volumetric room andoutside prior (Section 4.7), weighted by the volume of cell c .Note that this term is included with a negative sign within F C such that its value is being maximized. The wall face cost terms W F b and W F i penalize placement of walls in terms of the re-quired boundary and inner face areas, respectively. This penaltyis attenuated for faces with high support. The terms are definedas W F b : = (cid:88) f ca , cb ∈F (cid:88) w ∈W cacb x c b , w · (1 − p F ( f c a , c b )) · area ( f c a , c b ) , (9)and W F i : = (cid:88) f ca , cb ∈F (cid:88) w ∈W cacb ( x c b , w − x c a , w ) · (1 − p F ( f c a , c b )) · area ( f c a , c b ) , (10)where x c a , w , x c b , w are the binary variables for the assignment ofthe wall label w to the cells c a , c b respectively, p F ( f c a , c b ) is theface support prior (Section 4.7), and area ( f c a , c b ) is the area offace f c a , c b . It should be noted that ( x c b , w − x c a , w ) ∈ { , } due toConstraint 5. Also note that in Equation 9, it su ffi ces to consider x c b , w since for a boundary face of wall w , x c a , w does not exist (i.e. x c a , w can be considered to be zero).We then minimize F C s.t. Constraints 1-6 using the GurobiOptimizer [44].Note that in our experiments, we added the following con-straint which gave a small performance improvement althoughit is already implied by Constraints 1-2. At boundary faces ofoutside area, the outside label may only occur on the negative side of the separating face, i.e. ∀ f c a , c b ∈ F : x c a , o − x c b , o ≤ . (11)We attribute this slight performance improvement to heuristicsused by the particular optimizer implementation. The result is an assignment of each cell to either one room, orthe outside area. Cells which are assigned the outside area mayalso be assigned a nonempty set of walls. On the one hand thisprovides a dense segmentation of space into rooms and outsidespace. Volumes to which multiple walls are assigned are (vol-umetric) intersections of the respective walls. Since the under-lying data structure provides adjacency information between all reprint / ISPRS Journal of Photogrammetry and Remote Sensing (2019) 9 cells, semantic information like room adjacency and wall inci-dence is immediately available, e.g. for navigation or simulationpurposes. On the other hand this information is closely relatedto the definition of building elements in BIM formats like IFC.This enables immediate transfer of the results into standard ar-chitecture software and integration into existing BIM pipelines.
5. Implementation details
Input point clouds were subsampled to a minimum point dis-tance of 2 cm. Plane detection was performed using a planedistance threshold of 1 cm, a point cluster epsilon of 20 cm,a normal threshold of about 6 ◦ (18 ◦ for the “Case study 2”dataset), minimum support of 1000 points and miss probabil-ity of 0.001. Multi-label bitmaps had a resolution (pixel size)of 10 cm, occupancy bitmaps had a resolution of 20 cm. Threeray casting iterations were performed for point cloud cleaning.For automatic labeling, MCL was used with default parameters(inflation set to 2.0) in multi-threaded mode. The surface costweight α in Equation 7 was empirically chosen as 0.04. Weused PCL 1.8.1 [45], CGAL 4.12 [42, 46], MCL 14-137 [41],Gurobi 8.0.1 [44], and NVIDIA OptiX 5.0 for GPU-based raycasting under Linux on a 6-core Intel i7 CPU and a NVIDIAGeForce GTX 980 GPU.
6. Evaluation
We evaluate the reconstruction quality and performance ofour approach on a variety of datasets and show comparisonswith groundtruth IFC and related work. Furthermore, we exem-plify the flexibility of our integer linear programming approachby specifying additional constraints to modify and guide the re-sulting reconstruction in an intuitive manner.
Datasets.
We used a variety of real-world datasets and one syn-thetic dataset for our evaluation. Table 2 shows six multi-storypoint clouds measured using terrestrial laser scanners. Thesedatasets were provided by The Royal Danish Academy of FineArts Schools of Architecture, Design and Conservation (CITA).The Table lists properties of the input data including the numberof points and scans, as well as quantities derived during recon-struction such as the number of room labels, extracted surfaces,wall candidates, etc. It also shows runtime measurements of themain processing steps. We also tested our approach on publiclyavailable datasets provided by other research groups. Figure 5shows the dataset “synth3” by the Visualization and MultiMe-dia Lab at University of Zurich, Figure 10 depicts the dataset“Case study 2” from the ISPRS Benchmark on Indoor Mod-eling [47], and Figure 9 shows the dataset “Area 3” from theStanford 3D Large-Scale Indoor Spaces Dataset [48]. We usedthe latter two for demonstrating di ff erent parameters and inter-active modification as described below. Reconstruction quality.
Our reconstruction approach generallyworked well on the test datasets without any dataset-specifictuning. Automatic outlier removal reliably ignored even large-scale clutter scanned through windows in e.g. Datasets 1, 3, and6. In some cases, particularly thick walls (bottom region of
Fig. 5. Our reconstruction result on the synthetic dataset “synth3” by theVisualization and MultiMedia Lab at University of Zurich. Rooms andwalls are accurately reconstructed.Fig. 6. Comparison of a hand-crafted BIM model (left) and our recon-struction (right) of Dataset 5 (see Table 2). Reconstructed room labels weremanually overlaid on the BIM model for reference.
Dataset 1, top region of Dataset 2) were reconstructed as twothinner, parallel wall elements which may be a matter of inter-pretation. Increasing the maximum thickness of generated wallcandidates in these cases can help recognizing such cases assingle walls. A few cases of room-oversegmentation can be ob-served. In Dataset 4, the large central room is split into a largerL-shaped part (orange) and a smaller room (green, to the rightof the building) without a real wall separating the reconstructedrooms in the point cloud data. In Dataset 5, indentations ofthe central room (orange) were reconstructed as small, separaterooms (cyan, purple). Since our approach currently only con-siders horizontal ceilings, the slanted ceiling of the staircase inDataset 6 (yellow, elongated room) is reconstructed as a hori-zontal structure (see also Limitations below).
Runtime.
Total runtime for the reconstruction of the testdatasets lies in the range of one minute (Datasets 1, 2) to10 minutes (Dataset 6). The runtime of primitive detectionis mainly dependent on the CGAL implementation, and thetime for solving the optimization problem is the runtime of theGurobi optimizer. The runtime for auto labeling contains thetime for our raycasting and clustering using the Markov ClusterAlgorithm. Runtime of the optimization mainly depends on thecomplexity of the plane arrangement, which in turn dependson the number of detected surfaces since every surface intro-duces global splits in the cell complex. Therefore a tradeo ff between reconstructing details (i.e. small surfaces) and compu-tational feasibility must be made. In our experiments, we thuschose a minimum estimated area of 2 m for vertical surfacesand 5 m for horizontal surfaces. Comparison to IFC.
For Dataset 5 a corresponding, profes-sionally made BIM model in IFC format was available. Figure6 shows a comparison between our reconstruction and the BIM / ISPRS Journal of Photogrammetry and Remote Sensing (2019)
Dataset 1 Dataset 2 Dataset 3
Input / / / / / / / / Room labels / Surfaces / Walls / + / + / + / + / + / + / Variables / Constraints 17666 / / / / / / Runtime (seconds)
Plane detection 18.2 20.1 73.9Cleaning (3 iterations) 14.1 21.9 32.7Auto labeling 6.3 7.2 11.4Arrangement + Priors 14.6 11.5 15.5Optimization 20.9 7.4 9.8Dataset 4 Dataset 5 Dataset 6
Input / / / / / / / / Room labels / Surfaces / Walls / + / + / + / + / + / + / Variables / Constraints 42262 / / / / / / Runtime (seconds)
Plane detection 51.1 82.6 218.6Cleaning (3 iterations) 28.2 59.0 170.9Auto labeling 16.1 56.8 84.5Arrangement + Priors 38.9 29.1 53.6Solving 85.4 42.9 182.8
Table 2. Evaluation results on various real-world datasets. The number of scans in the point cloud is shown for reference only and is not used in ourapproach. The number of points is the point count after subsampling. The number of surfaces and walls is given as vertical + horizontal . reprint / ISPRS Journal of Photogrammetry and Remote Sensing (2019) 11
Fig. 7. Comparison between di ff erent reconstruction approaches. Left: Input point cloud viewed from above. Center: The method described in [5] mayfail to regularize chains of almost coplanar walls, leading to segmented walls (circles). Also, reliance on separate scans as initial room labeling may lead tooversegmented rooms (dashed rectangle). Right: Our approach overcomes these issues by incorporating costs for all surfaces of volumetric wall elements,and room segmentation that is independent of scan positions.Fig. 8. Additional constraints may be added to interactively steer the re-construction. Small indentations of wall surfaces (left) are initially lostin the reconstruction. By forcing regions to be outside area (center), ourmethod finds an alternative wall placement under these constraints (right). model. Colors of the reconstructed rooms were manually over-laid on the IFC model on the left-hand side. All rooms that werepart of the scans mostly match the groundtruth BIM. The upperstory of the building is connected to the lower story througha large horizontal opening. These areas were reconstructed astwo separate rooms (red and orange) and the railing at the edgeof the gallery was reconstructed as walls. The small cyan andpurple rooms are an oversegmentation of the upper floor, prob-ably due to the dilated surface support. However, this error caneasily be fixed manually. Comparison to related work.
A comparison between recon-structions by the approach described in [5] and our method isshown in Figure 7. In addition to fundamental advantages of ourapproach such as reconstruction of multiple stories, two crucialdi ff erences are particularly notable. First, our approach resultsin stronger regularization of wall elements where using multi-ple di ff erent, similar walls to represent the building would beunnecessary. The approach in [5] leads to jumps between dif-ferent, almost coplanar walls (Figure 7, center, black circles) in-stead of using longer, continuous walls. This can be explainedby the principle that the approach separated rooms by wall cen-ter lines in 2D such that jumping from one wall to an almostcoplanar wall resulted in almost no penalty in the cost function.In our case, a fully volumetric wall element would need to beadded to the model to connect the parallel walls, resulting in rel-atively high costs. Second, the approach in [5] relies on given,separate scans and their positions for estimating an initial roomsegmentation. This leads to an oversegmentation of the hallway(Figure 7, center, dashed rectangle) since it tries to reconstructone room per scan. Our method works independently of sepa-rate scans and estimates a room segmentation by unsupervised Fig. 9. Manual addition of a wall demonstrated on the dataset “Area 3”from the Stanford 3D Large-Scale Indoor Spaces Dataset. Top right: Ahallway ends without a terminating wall surface such that no wall candi-date is available for enclosing the protrusion. Reconstruction and pointcloud are shown overlaid. Bottom left: A wall candidate can easily beadded by drawing a line, adding two opposing “virtual” wall surfaces. Bot-tom right: The algorithm now encloses the protrusion, using the perpen-dicular walls with real support in the point cloud. clustering.
Interactive modification.
Our linear programming approach al-lows for additional constraints to be easily added. One exam-ple for manual post-processing of the reconstruction results byinteractively adding hard constraints is shown in Figure 8. Inthis case the indentation of the wall surfaces on the left andright sides of the building were lost by the regularization of themodel as can be seen in Figure 8, center. The user has the op-tion to add constraints such as forcing inside area, outside area,wall, no wall, etc. by clicking at the desired location. In thiscase, the highlighted locations were forced to be outside area.The algorithm then finds the next best option, placing new wallsthat fulfill all constraints as shown in Figure 8, right. Anotherexample is shown in Figure 9 where a hallway ends without anyterminating wall surface in the input data. Since the algorithmhas no wall candidate available, it cannot enclose the protrud-ing room area. By adding a virtual wall candidate by meansof simply drawing a line, the algorithm is able to include theprotrusion in the reconstructed model, automatically using theperpendicular wall surfaces that are present in the input data.Di ff erent choices of the wall surface penalty weight α in Equa- / ISPRS Journal of Photogrammetry and Remote Sensing (2019)
Fig. 10. Di ff erent settings for the wall surface cost parameter α in Equation 7 demonstrated on the dataset “Case study 2” from the ISPRS Benchmark onIndoor Modeling. Center: Our default setting of α = . results in some walls to be fitted to windows which have high point support in this dataset. Also,a slab has a hole since floor support in staircases is often complex. Right: Increasing to α = . leads to stronger regularization of walls and slabs. tion 7 control global regularization strength. Figure 10, centershows a reconstruction where some wall and slab elements areslightly misplaced due to relatively strong surface support atwindows. Increasing α from our default of 0.04 to 0.08 leads tostronger regularization as shown in Figure 10, right. Limitations.
One technical limitation of our current implemen-tation is that slanted walls, floors or ceilings are not taken intoaccount although this is not an inherent limitation of our ap-proach. The reason for our decision not to include these ele-ments is that the construction of the 3D cell complex needs tobe exact to guarantee the integrity of the data structure (e.g.cell neighborhood). Unfortunately, computing the cell com-plex in 3D induces numerical problems and currently we donot have a stable implementation for this task at our disposal.We thus opted to use the numerically stable implementation of2D arrangements in CGAL [46] and extend it to 3D by stack-ing 2D arrangements separated by horizontal planes. A nu-merically stable extension of arrangements supporting generalslanted planes would be an interesting direction for future re-search which we consider to be outside the scope of this paper.Processing of very large datasets may also require further op-timizations to make them computationally feasible. In particu-lar, using a global plane arrangement results in a large increaseof cells and thus variables in the optimization model with ev-ery additional detected surface. More sophisticated selectionof potential surfaces, and improved optimization methods, e.g.splitting the problem into smaller subproblems, are targets forfurther research. Last but not least, our current algorithm is notable to identify and include important architectural structuresoverarching the whole building like the pillars that are includedin the hand-crafted model in Figure 6. Automatically identi-fying such structural elements and incorporating them into theautomatic reconstruction is also an interesting direction for fu-ture research.
7. Conclusion and future work
We have presented a novel approach to tackle the indoorbuilding reconstruction problem from point clouds using inte-ger linear programming. In contrast to previous methods, ourapproach reconstructs fully volumetric, interconnected wall en-tities and room topology on multi-story buildings with weakassumptions on the input data. The resulting models are veryclose to the requirements needed for Building Information Modeling tasks including volumetric representations of roomspaces and wall entities, and their interrelations. Additionalhard constraints such as forcing or avoiding certain entities atchosen locations may simply be added as constraints of the op-timization problem. We demonstrated our approach on a varietyof real-world datasets.Future work for our proposed method includes the extensionof the plane arrangement data structure to support slanted sur-faces and possibly non-planar primitives. Strategies for reduc-ing computational complexity by e.g. pruning invalid surfaceand wall candidates early in the process would improve appli-cability to larger-scale datasets. Also, connecting our recon-struction methodology with e.g. opening and object detectionapproaches would further enrich the resulting models.
Acknowledgments
We acknowledge the Visualization and MultiMedia Lab at University ofZurich (UZH) and Claudio Mura for the acquisition of the 3D point clouds, andUZH as well as ETH Zrich for their support to scan the rooms represented inthese datasets. Their datasets were used in our evaluation (Figure 5). We alsoused datasets provided by The Royal Danish Academy of Fine Arts Schoolsof Architecture, Design and Conservation (CITA) (Table 2), from The ISPRSBenchmark on Indoor Modeling [47] (Figure 10), and from the Stanford 3DLarge-Scale Indoor Spaces Dataset [48] (Figure 9). This work was supported bythe DFG projects KL 1142 / / References [1] Sanchez, V, Zakhor, A. Planar 3D modeling of building interiors frompoint cloud data. In: Image Processing (ICIP), 2012 19th IEEE Interna-tional Conference on. IEEE; 2012, p. 1777–1780.[2] Oesau, S, Lafarge, F, Alliez, P. Indoor scene reconstruction usingfeature sensitive primitive extraction and graph-cut. ISPRS Journal ofPhotogrammetry and Remote Sensing 2014;90:68–82.[3] Turner, E, Cheng, P, Zakhor, A. Fast, automated, scalable generationof textured 3D models of indoor environments. IEEE Journal of SelectedTopics in Signal Processing 2015;9(3):409–421.[4] Mura, C, Mattausch, O, Pajarola, R. Piecewise-planar reconstructionof multi-room interiors with arbitrary wall arrangements. In: ComputerGraphics Forum; vol. 35. Wiley Online Library; 2016, p. 179–188.[5] Ochmann, S, Vock, R, Wessel, R, Klein, R. Automatic reconstructionof parametric building models from indoor point clouds. Computers &Graphics 2016;54:94–103. URL: . doi: doi:10.1016/j.cag.2015.07.008 ; special Issue on CAD / Graphics 2015.reprint / ISPRS Journal of Photogrammetry and Remote Sensing (2019) 13[6] Murali, S, Speciale, P, Oswald, MR, Pollefeys, M. Indoor scan2bim:Building information models of house interiors. In: Intelligent Robotsand Systems (IROS), 2017 IEEE / RSJ International Conference on. IEEE;2017, p. 6126–6133.[7] Liu, C, Wu, J, Furukawa, Y. Floornet: A unified framework for floorplanreconstruction from 3D scans. arXiv preprint arXiv:180400090 2018;.[8] Budroni, A, Boehm, J. Automated 3D reconstruction of interiorsfrom point clouds. International Journal of Architectural Computing2010;8(1):55–73.[9] Adan, A, Huber, D. 3D reconstruction of interior wall surfaces under oc-clusion and clutter. In: 3D Imaging, Modeling, Processing, Visualizationand Transmission (3DIMPVT), 2011 International Conference on. IEEE;2011, p. 275–281.[10] Xiong, X, Adan, A, Akinci, B, Huber, D. Automatic creation of se-mantically rich 3D building models from laser scanner data. Automationin Construction 2013;31:325–337.[11] Mura, C, Mattausch, O, Villanueva, AJ, Gobbetti, E, Pajarola, R. Auto-matic room detection and reconstruction in cluttered indoor environmentswith complex room layouts. Computers & Graphics 2014;44:20–32.[12] Previtali, M, Barazzetti, L, Brumana, R, Scaioni, M. Towards automaticindoor reconstruction of cluttered building rooms from point clouds. IS-PRS Annals of the Photogrammetry, Remote Sensing and Spatial Infor-mation Sciences 2014;2(5):281.[13] Ambrus¸, R, Claici, S, Wendt, A. Automatic room segmentation fromunstructured 3-d data of indoor environments. IEEE Robotics and Au-tomation Letters 2017;2(2):749–756.[14] Macher, H, Landes, T, Grussenmeyer, P. From point clouds to build-ing information models: 3D semi-automatic reconstruction of indoors ofexisting buildings. Applied Sciences 2017;7(10):1030.[15] Wang, R, Xie, L, Chen, D. Modeling indoor spaces using decompositionand reconstruction of structural elements. Photogrammetric Engineering& Remote Sensing 2017;83(12):827–841.[16] Okorn, B, Xiong, X, Akinci, B, Huber, D. Toward automated modelingof floor plans. In: Proceedings of the symposium on 3D data processing,visualization and transmission; vol. 2. 2010,.[17] Liu, C, Wu, J, Kohli, P, Furukawa, Y. Raster-to-vector: Revisitingfloorplan transformation. In: Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition. 2017, p. 2195–2203.[18] Oesau, S, Lafarge, F, Alliez, P. Indoor scene reconstruction usingprimitive-driven space partitioning and graph-cut. In: Eurographics work-shop on urban data modelling and visualisation. 2013,.[19] Xiao, J, Furukawa, Y. Reconstructing the worlds museums. Internationaljournal of computer vision 2014;110(3):243–258.[20] Mura, C, Mattausch, O, Villanueva, AJ, Gobbetti, E, Pajarola,R. Robust reconstruction of interior building structures with multiplerooms under clutter and occlusions. In: Computer-Aided Design andComputer Graphics (CAD / Graphics), 2013 International Conference on.IEEE; 2013, p. 52–59.[21] Mura, C, Villanueva, AJ, Mattausch, O, Gobbetti, E, Pajarola, R.Reconstructing complex indoor environments with arbitrary wall orienta-tions. In: Eurographics (Posters). 2014, p. 19–20.[22] Turner, E, Zakhor, A. Floor plan generation and room labeling of indoorenvironments from laser range data. In: Computer Graphics Theory andApplications (GRAPP), 2014 International Conference on. IEEE; 2014,p. 1–12.[23] Mura, C, Pajarola, R. Exploiting the room structure of buildings forscalable architectural modeling of interiors. In: ACM SIGGRAPH 2017Posters. ACM; 2017, p. 4.[24] Stambler, A, Huber, D. Building modeling through enclosure reasoning.In: 3D Vision (3DV), 2014 2nd International Conference on; vol. 2. IEEE;2014, p. 118–125.[25] Thomson, C, Boehm, J. Automatic geometry generation from pointclouds for BIM. Remote Sensing 2015;7(9):11753–11775.[26] Monszpart, A, Mellado, N, Brostow, GJ, Mitra, NJ. Rapter: rebuildingman-made scenes with regular arrangements of planes. ACM Trans Graph2015;34(4):103–1.[27] Nan, L, Wonka, P. Polyfit: Polygonal surface reconstruction frompoint clouds. In: The IEEE International Conference on Computer Vi-sion (ICCV). 2017,.[28] Jung, J, Stachniss, C, Kim, C. Automatic room segmentation of 3Dlaser data using morphological processing. ISPRS International Journalof Geo-Information 2017;6(7):206. [29] Bobkov, D, Kiechle, M, Hilsenbeck, S, Steinbach, E. Room segmenta-tion in 3D point clouds using anisotropic potential fields. In: Multimediaand Expo (ICME), 2017 IEEE International Conference on. IEEE; 2017,p. 727–732.[30] Ochmann, S, Vock, R, Wessel, R, Tamke, M, Klein, R. Automaticgeneration of structural building descriptions from 3D point cloud scans.In: Computer Graphics Theory and Applications (GRAPP), 2014 Inter-national Conference on. IEEE; 2014, p. 1–8.[31] Bassier, M, Van Genechten, B, Vergauwen, M. Classification of sensorindependent point cloud data of building objects using random forests.Journal of Building Engineering 2018;.[32] Ad´an, A, Quintana, B, Prieto, SA, Bosch´e, F. Scan-to-bimfor secondarybuilding components. Advanced Engineering Informatics2018;37:119–138.[33] Son, H, Kim, C. Semantic as-built 3D modeling of structural elements ofbuildings based on local concavity and convexity. Advanced EngineeringInformatics 2017;34:114–124.[34] Garwood, TL, Hughes, BR, O’Connor, D, Calautit, JK, Oates, MR,Hodgson, T. A framework for producing gbxml building geometry frompoint clouds for accurate and e ffi cient building energy modelling. AppliedEnergy 2018;224:527–537.[35] Hyland, N, OKee ff e, S, Dore, C, Brodie, S. Automatic open standardreporting for dimensional control compliance 2017;.[36] OKee ff e, S, Hyland, N, Dore, C, Brodie, S. Automatic validation ofas-is and as-generated ifc bims for advanced scan-to-bim methods 2017;.[37] Brodie, S, Hyland, N, Dore, C, OKee ff e, S. The bim & scan R (cid:13) plat-form: A cloud-based cyber-physical system for automated solutions util-ising real & virtual worlds. 2017;.[38] Krispel, U, Evers, HL, Tamke, M, Ullrich, T. Data completion inbuilding information management: electrical lines from range scans andphotographs. Visualization in Engineering 2017;5(1):4.[39] Krijnen, T, Beetz, J. An ifc schema extension and binary serializationformat to e ffi ciently integrate point cloud data into building models. Ad-vanced Engineering Informatics 2017;33:473–490.[40] Schnabel, R, Wahl, R, Klein, R. E ffi cient ransac for point-cloud shapedetection. Computer Graphics Forum 2007;26(2):214–226.[41] Dongen, S. A cluster algorithm for graphs. Tech. Rep.; Amsterdam, TheNetherlands, The Netherlands; 2000.[42] Oesau, S, Verdie, Y, Jamin, C, Alliez, P, Lafarge, F, Giraudot, S. Pointset shape detection. In: CGAL User and Reference Manual; 4.12 ed.CGAL Editorial Board; 2018,URL: https://doc.cgal.org/4.12/Manual/packages.html .[43] Boykov, Y, Veksler, O, Zabih, R. Fast approximate energy minimiza-tion via graph cuts. IEEE Transactions on pattern analysis and machineintelligence 2001;23(11):1222–1239.[44] Gurobi Optimization, I. Gurobi optimizer reference manual. 2016. URL: .[45] Rusu, RB, Cousins, S. 3D is here: Point cloud library (pcl). In: Roboticsand automation (ICRA), 2011 IEEE International Conference on. IEEE;2011, p. 1–4.[46] Wein, R, Berberich, E, Fogel, E, Halperin, D, Hemmer, M, Salz-man, O, et al. 2D arrangements. In: CGAL User and Reference Manual;4.12 ed. CGAL Editorial Board; 2018,URL: https://doc.cgal.org/4.12/Manual/packages.html .[47] Khoshelham, K, Vilari˜no, LD, Peter, M, Kang, Z, Acharya, D. Theisprs benchmark on indoor modelling. International Archives of the Pho-togrammetry, Remote Sensing & Spatial Information Sciences 2017;42.[48] Armeni, I, Sax, A, Zamir, AR, Savarese, S. Joint 2D-3D-Semantic Data for Indoor Scene Understanding. ArXiv e-prints2017; arXiv:1702.01105arXiv:1702.01105