Franz Leberl

Robust active appearance models and their application to medical image analysis

Active appearance models (AAMs) have been successfully used for a variety of segmentation tasks in medical image analysis. However, gross disturbances of objects can occur in routine clinical setting caused by pathological changes or medical interventions. This poses a problem for AAM-based segmentation, since the method is inherently not robust. In this paper, a novel robust AAM (RAAM) matching algorithm is presented. Compared to previous approaches, no assumptions are made regarding the kind of gray-value disturbance and/or the expected magnitude of residuals during matching. The method consists of two main stages. First, initial residuals are analyzed by means of a mean-shift-based mode detection step. Second, an objective function is utilized for the selection of a mode combination not representing the gross outliers. We demonstrate the robustness of the method in a variety of examples with different noise conditions. The RAAM performance is quantitatively demonstrated in two substantially different applications, diaphragm segmentation and rheumatoid arthritis assessment. In all cases, the robust method shows an excellent behavior, with the new method tolerating up to 50% object area covered by gross gray-level disturbances.

3D MURALE: a multimedia system for archaeology

This paper introduces the 3D Measurement and Virtual Reconstruction of Ancient Lost Worlds of Europe system (3D MURALE). It consists of a set of tools for recording, reconstructing, encoding, visualising and database searching/querying that operate on buildings, building parts, statues, statue parts, pottery, stratigraphy, terrain geometry and texture and material texture. The tools are loosely linked together by a common database on which they all have the facility to store and access data. The paper describes the overall architecture of the 3D MURALE system and then briefly describes the functionality of the tools provided by the project. The paper compares the multimedia studio architecture adopted in this project with other multimedia studio architectures.

Diaphragm dome surface segmentation in CT data sets: a 3D active appearance model approach

Knowledge about the location of the diaphragm dome surface, which separates the lungs and the heart from the abdominal cavity, is of vital importance for applications like automated segmentation of adjacent organs (e.g., liver) or functional analysis of the respiratory cycle. We present a new 3D Active Appearance Model (AAM) approach to segmentation of the top layer of the diaphragm dome. The 3D AAM consists of three parts: a 2D closed curve (reference curve), an elevation image and texture layers. The first two parts combined represent 3D shape information and the third part image intensity of the diaphragm dome and the surrounding layers. Differences in height between dome voxels and a reference plane are stored in the elevation image. The reference curve is generated by a parallel projection of the diaphragm dome outline in the axial direction. Landmark point placement is only done on the (2D) reference curve, which can be seen as the bounding curve of the elevation image. Matching is based on a gradient-descent optimization process and uses image intensity appearance around the actual dome shape. Results achieved in 60 computer generated phantom data sets show a high degree of accuracy (positioning error -0.07+/-1.29 mm). Validation using real CT data sets yielded a positioning error of -0.16+/-2.95 mm. Additional training and testing on in-vivo CT image data is ongoing.

Computer-aided liver surgery planning: an augmented reality approach

Surgical resection of liver tumors requires a detailed three-dimensional understanding of a complex arrangement of vasculature, liver segments and tumors inside the liver. In most cases, surgeons need to develop this understanding by looking at sequences of axial images from modalities like X-ray computed tomography. A system for liver surgery planning is reported that enables physicians to visualize and refine segmented input liver data sets, as well as to simulate and evaluate different resections plans. The system supports surgeons in finding the optimal treatment strategy for each patient and eases the data preparation process. The use of augmented reality contributes to a user-friendly design and simplifies complex interaction with 3D objects. The main function blocks developed so far are: basic augmented reality environment, user interface, rendering, surface reconstruction from segmented volume data sets, surface manipulation and quantitative measurement toolkit. The flexible design allows to add functionality via plug-ins. First practical evaluation steps have shown a good acceptance. Evaluation of the system is ongoing and future feedback from surgeons will be collected and used for design refinements.

Shape- and appearance-based segmentation of volumetric medical images

A novel approach to three-dimensional segmentation, parametric surface representation, and interactive surface modification is reported. The combination of shape-based and appearance-based volumetric segmentation, parametric representation of 3D surfaces, and their interactive modification and editing forms a very powerful paradigm for a variety of volumetric segmentation tasks. Performance assessment of the method for automated segmentation of diaphragm surfaces in volumetric 3D CT images is ongoing.

High-quality texture reconstruction from multiple views†

This paper presents a method to automatically calculate texture maps for a given three-dimensional object out of a sequence of images. It is used in our image-based modeling approach after the registration of the images and the geometric modeling is done. We show that the presented method uses the information from all images by implicitly applying a weighting function. Using this approach the consideration of modeled occlusions as well as the detection and removal of non-modeled occlusions is accomplished. The final resolution of the texture maps can be adjusted on a pixel/cm basis. Copyright

3-Dimensional Building Details from Aerial Photography for Internet Maps

This paper introduces the automated characterization of real estate (real property) for Internet mapping. It proposes a processing framework to achieve this task from vertical aerial photography and associated property information. A demonstration of the feasibility of an automated solution builds on test data from the Austrian City of Graz. Information is extracted from vertical aerial photography and various data products derived from that photography in the form of a true orthophoto, a dense digital surface model and digital terrain model, and a classification of land cover. Maps of cadastral property boundaries aid in defining real properties. Our goal is to develop a table for each property with descriptive numbers about the buildings, their dimensions, number of floors, number of windows, roof shapes, impervious surfaces, garages, sheds, vegetation, presence of a basement floor, and other descriptors of interest for each and every property of a city. From aerial sources, at a pixel size of 10 cm, we show that we have obtained positional accuracies in the range of a single pixel, an accuracy of areas in the 10% range, floor counts at an accuracy of 93% and window counts at 86% accuracy. We also introduce 3D point clouds of facades and their creation from vertical aerial photography, and how these point clouds can support the definition of complex facades.

Multiresolution Texture for Photorealistic Rendering

This paper presents a method to automatically calculate texture maps for a given 3D object from a sequence of images. It is used in our image-based modeling approach after the registration of the images and the geometric modeling has been done. We show that the presented method uses the information from all images by implicitly applying a weighting function. Using this approach, the consideration of modeled occlusions as well as the detection and removal of non-modeled occlusions is accomplished. The final resolution of the texture maps can be adjusted on a pixel/cm basis.

Aerial Computer Vision for a 3D Virtual Habitat

An Internet-embedded 3D model of a human habitat is feasible and useful. In lieu of a 2D Earth map, the authors describe a 3D model with human-scale objects in urban spaces and inside buildings. Here, they focus on information from aerial imagery.

Computer Aided Liver Surgery Planning Based on Augmented Reality Techniques

A system for liver surgery planning is reported that enables physicians to visualize and refine segmented input liver data sets, as well as to simulate and evaluate different resection plans. The system supports surgeons in finding the optimal treatment strategy for each patient and facilitates the data preparation process. Using augmented reality eases complex interaction with 3D objects and contributes to a user-friendly design. First practical evaluation steps have shown a good acceptance. Evaluation of the system is ongoing and future feedback from surgeons will be collected and used for design refinements.