David Koller

The digital Michelangelo project: 3D scanning of large statues

We describe a hardware and software system for digitizing the shape and color of large fragile objects under non-laboratory conditions. Our system employs laser triangulation rangefinders, laser time-of-flight rangefinders, digital still cameras, and a suite of software for acquiring, aligning, merging, and viewing scanned data. As a demonstration of this system, we digitized 10 statues by Michelangelo, including the well-known figure of David, two building interiors, and all 1,163 extant fragments of the Forma Urbis Romae, a giant marble map of ancient Rome. Our largest single dataset is of the David - 2 billion polygons and 7,000 color images. In this paper, we discuss the challenges we faced in building this system, the solutions we employed, and the lessons we learned. We focus in particular on the unusual design of our laser triangulation scanner and on the algorithms and software we developed for handling very large scanned models.

Real-time, continuous level of detail rendering of height fields

We present an algorithm for real-time level of detail reduction and display of high-complexity polygonal surface data. The algorithm uses a compact and efficient regular grid representation, and employs a variable screen-space threshold to bound the maximum error of the projected image. A coarse level of simplification is performed to select discrete levels of detail for blocks of the surface mesh, followed by further simplification through repolygonalization in which individual mesh vertices are considered for removal. These steps compute and generate the appropriate level of detail dynamically in real-time, minimizing the number of rendered polygons and allowing for smooth changes in resolution across areas of the surface. The algorithm has been implemented for approximating and rendering digital terrain models and other height fields, and consistently performs at interactive frame rates with high image quality.

Protected interactive 3D graphics via remote rendering

Valuable 3D graphical models, such as high-resolution digital scans of cultural heritage objects, may require protection to prevent piracy or misuse, while still allowing for interactive display and manipulation by a widespread audience. We have investigated techniques for protecting 3D graphics content, and we have developed a remote rendering system suitable for sharing archives of 3D models while protecting the 3D geometry from unauthorized extraction. The system consists of a 3D viewer client that includes low-resolution versions of the 3D models, and a rendering server that renders and returns images of high-resolution models according to client requests. The server implements a number of defenses to guard against 3D reconstruction attacks, such as monitoring and limiting request streams, and slightly perturbing and distorting the rendered images. We consider several possible types of reconstruction attacks on such a rendering server, and we examine how these attacks can be defended against without excessively compromising the interactive experience for non-malicious users.

A methodology for the evaluation of travel techniques for immersive virtual environments

We present a framework for the analysis and evaluation oftravel, or viewpoint motion control, techniques for use in immersive virtual environments (VEs). In previous work, we presented a taxonomy of travel techniques and a set of experiments mapping parts of the taxonomy to various performance metrics. Since these initial experiments, we have expanded the framework to allow evaluation of not only the effects of different travel techniques, but also the effects of many outside factors simultaneously. Combining this expanded framework with the measurement of multiple response variables epitomises the philosophy oftestbed evaluation. This experimental philosophy leads to a deeper understanding of the interaction and the technique(s) in question, as well as to broadly generalisable results. We also present an example experiment within this expanded framework, which evaluates the users ability to gather information while travelling through a virtual environment. Results indicate that, of the variables tested, the complexity of the environment is by far the most important factor.

Research challenges for digital archives of 3D cultural heritage models

The increasing creation of 3D cultural heritage models has resulted in a need for the establishment of centralized digital archives. We advocate open repositories of scientifically authenticated 3D models based on the example of traditional scholarly journals, with standard mechanisms for preservation, peer review, publication, updating, and dissemination of the 3D models. However, fully realizing this vision will require addressing a number of related research challenges. In this article, we first give a brief background of the virtual heritage discipline, and characterize the need for centralized 3D archives, including a preliminary needs assessment survey of virtual heritage practitioners. Then we describe several existing 3D cultural heritage repositories, and enumerate a number of technical research challenges that should be addressed to realize an ideal archive. These challenges include digital rights management for the 3D models, clear depiction of uncertainty in 3D reconstructions, version control for 3D models, effective metadata structures, long-term preservation, interoperability, and 3D searching. Other concerns are provision for the application of computational analysis tools, and the organizational structure of a peer-reviewed 3D model archive.

Head-tracked orbital viewing: an interaction technique for immersive virtual environments

An interaction technique for immersive virtual environments called “head-tracked orbital viewing” is described. The user’s head orientation is tracked and mapped so as to move the viewpoint of the user about the surface of a virtual sphere surrounding a center of rotation. The technique is useful for object examination tasks in a virtual world, allowing the user to quickly and easily view an object from many perspectives.

Remote rendering for ultrascale data

The mission of the SciDAC Institute for Ultrascale Visualization is to address the upcoming petascale visualization challenges. As we move to petascale computation, we are seeing a trend not only in the growth but also in the consolidation of computing resources. As the distances between user and petascale visualization resources grow, the expected performance of the network degrades, especially with respect to latency. In this paper we will explore a technique for remote visualization that leverages unstructured lumigraph rendering. This technique will provide an interactive rendering experience regardless of the network performance to the remote visualization resource. The unstructured lumigraph rendering can also replace many of the other level-of-detail techniques currently used that have problems that are exasperated by petascale data.

Fast Omnidirectional 3D Scene Acquisition with an Array of Stereo Cameras

We present an omnidirectional 3D acquisition system based on a mobile array of high-resolution consumer digital SLR cameras that automatically capture high dynamic range stereo pairs across a full 360-degree panorama. The stereo pairs are augmented with a time-varying lighting pattern created using standard photographic flashes, lenses, and patterned slides. Spacetime stereo techniques are used to generate 3D range images with corresponding color data from the HDR photographs. The multiple range images are aligned with egomotion estimation and ICP registration techniques, and volumetric merging and color texturing algorithms allow the rapid creation of complete 3D models. The resulting system compares favorably with other state of the art 3D acquisition technologies in the resolution and quality of its output, and can be faster and less expensive than 3D laser scanners for digitizing large 3D scenes such as building interiors.

Quantifying usability in secure graphics: assessing the user costs of protecting 3D content

There is an increasing need for methods for secure dissemination of interactive 3D graphics content, providing protection for valuable 3D models while still allowing them to be widely shared. Existing systems for protected sharing of 3D models may introduce perturbations into the rendered images of the content, in order to defend against potential malicious reconstruction attacks that could otherwise recover the 3D model shape. However, the particular nature and magnitude of these perturbation defenses has not been based upon any rigorous analysis or measurement of their perceptual effect on non-malicious users of the protected graphics system. In this paper, we take the first steps toward such an analysis, conducting a series of user studies that evaluate the impact (as measured by user reaction time) of varying amounts of noise applied to user interactions in a real-time 3D rendering system. We are thus able to experimentally determine the most appropriate tradeoffs between noise perturbation defenses and the security of the 3D content against typical reconstruction attacks.