Julie Dorsey

Image-based modeling and photo editing

We present an image-based modeling and editing system that takes a single photo as input. We represent a scene as a layered collection of depth images, where each pixel encodes both color and depth. Starting from an input image, we employ a suite of user-assisted techniques, based on a painting metaphor, to assign depths and extract layers. We introduce two specific editing operations. The first, a “clone brushing tool,” permits the distortion-free copying of parts of a picture, by using a parameterization optimization technique. The second, a “texture-illuminance decoupling filter,” discounts the effect of illumination on uniformly textured areas, by decoupling large- and small-scale features via bilateral filtering. Our system enables editing from different viewpoints, extracting and grouping of image-based objects, and modifying the shape, color, and illumination of these objects.

Stable real-time deformations

The linear strain measures that are commonly used in real-time animations of deformable objects yield fast and stable simulations. However, they are not suitable for large deformations. Recently, more realistic results have been achieved in computer graphics by using Greens non-linear strain tensor, but the non-linearity makes the simulation more costly and introduces numerical problems.In this paper, we present a new simulation technique that is stable and fast like linear models, but without the disturbing artifacts that occur with large deformations. As a precomputation step, a linear stiffness matrix is computed for the system. At every time step of the simulation, we compute a tensor field that describes the local rotations of all the vertices in the mesh. This field allows us to compute the elastic forces in a non-rotated reference frame while using the precomputed stiffness matrix. The method can be applied to both finite element models and mass-spring systems. Our approach provides robustness, speed, and a realistic appearance in the simulation of large deformations.

Reconstructing 3D tree models from instrumented photographs

Our computer modeling technique reproduces a trees 3D volume and skeleton from instrumented photographs. The technique involves first constructing the skeleton (trunk and the major branches) of the tree, and then applying an L-system starting from this skeleton. L-systems are one of the better known procedural models in the graphics community, especially after their popularization by P. Prusinkiewicz and A. Lindenmayer (1990).

Conservative volumetric visibility with occluder fusion

Visibility determination is a key requirement in a wide range of graphics algorithms. This paper introduces a new approach to the computation of volume visibility, the detection of occluded portions of space as seen from a given region. The method is conservative and classifies regions as occluded only when they are guaranteed to be invisible. It operates on a discrete representation of space and uses the opaque interior of objects as occluders. This choice of occluders facilitates their extension into adjacent opaque regions of space, in essence maximizing their size and impact. Our method efficiently detects and represents the regions of space hidden by such occluders. It is the first one to use the property that occluders can also be extended into empty space provided this space is itself occluded from the viewing volume. This proves extremely effective for computing the occlusion by a set of occluders, effectively realizing occluder fusion. An auxiliary data structure represents occlusion in the scene and can then be queried to answer volume visibility questions. We demonstrate the applicability to visibility preprocessing for real-time walkthroughs and to shadow-ray acceleration for extended light sources in ray tracing, with significant acceleration in both cases.

Multi‐layered impostors for accelerated rendering

This paper describes the successful combination of pre‐generated and dynamically updated image‐based representations to accelerate the visualization of complex virtual environments. We introduce a new type of impostor, which has the desirable property of limiting de‐occlusion errors to a user‐specified amount. This impostor, composed of multiple layers of textured meshes, replaces the distant geometry and is much faster to draw. It captures the relevant depth complexity in the model without resorting to a complete sampling of the scene. We show that layers can be dynamically updated during visualization. This guarantees bounded scene complexity in each frame and also exploits temporal coherence to improve image quality when possible. We demonstrate the strengths of this approach in the context of city walkthroughs.

A procedural approach to authoring solid models

We present a procedural approach to authoring layered, solid models. Using a simple scripting language, we define the internal structure of a volume from one or more input meshes. Sculpting and simulation operators are applied within the context of the language to shape and modify the model. Our framework treats simulation as a modeling operator rather than simply as a tool for animation, thereby suggesting a new paradigm for modeling as well as a new level of abstraction for interacting with simulation environments.Capturing real-world effects with standard modeling techniques is extremely challenging. Our key contribution is a concise procedural approach for seamlessly building and modifying complex solid geometry. We present an implementation of our language using a flexible tetrahedral representation. We show a variety of complex objects modeled in our system using tools that interface with finite element method and particle system simulations.

Real-time simulation of deformation and fracture of stiff materials

Existing techniques for real-time simulation of object deformation are well suited for animating soft materials like human tissue or two-dimensional systems such as cloth. However, simulation of deformation in malleable materials and fracture in brittle materials has only been done offline because the underlying equations of motion are numericaly stiff, requiring many small steps in explicit integration schemes. In contrast, the better-behaved implicit integration techniques are more expensive per time step, particularly for volumetric meshes. We present a stable hybrid method for simulating deformation and fracture of materials in real-time. In our system, the effects of impact forces are computed only at discrete collision events. At these impacts, we treat objects as if they are anchored and compute their static equilibrium response using the Finite Element technique. Static analysis is not time-step bound and its stability is independent of the stiffness of the equations. The resulting deformations, or possible fractures, are computed based on internal stress tensors. Between collisions, disconnected objects are treated as rigid bodies. The simulator is demonstrated as part of a system that provides the user with physically-based tools to interactively manipulate 3D models.

Interactive Tone Mapping

Tone mapping and visual adaptation are crucial for the generation of static, photorealistic images. A largely unexplored problem is the simulation of adaptation and its changes over time on the visual appearance of a scene. These changes are important in interactive applications, including walkthroughs or games, where effects such as dazzling, slow dark-adaptation, or more subtle effects of visual adaptation can greatly enhance the immersive impression. In applications such as driving simulators, these changes must be modeled in order to reproduce the visibility conditions of real-world situations. In this paper, we address the practical issues of interactive tone mapping and propose a simple model of visual adaptation. We describe a multi-pass interactive rendering method that computes the average luminance in a first pass and renders the scene with a tone mapping operator in the second pass. We also propose several extensions to the tone mapping operator of Ferwerda et al. [FPSG96]. We demonstrate our model for the display of global illumination solutions and for interactive walkthroughs.

Context-aware textures

Interesting textures form on the surfaces of objects as the result of external chemical, mechanical, and biological agents. Simulating these textures is necessary to generate models for realistic image synthesis. The textures formed are progressively variant, with the variations depending on the global and local geometric context. We present a method for capturing progressively varying textures and the relevant context parameters that control them. By relating textures and context parameters, we are able to transfer the textures to novel synthetic objects. We present examples of capturing chemical effects, such as rusting; mechanical effects, such as paint cracking; and biological effects, such as the growth of mold on a surface. We demonstrate a user interface that provides a method for specifying where an object is exposed to external agents. We show the results of complex, geometry-dependent textures evolving on synthetic objects.

Sketching reality: Realistic interpretation of architectural designs

In this article, we introduce sketching reality, the process of converting a freehand sketch into a realistic-looking model. We apply this concept to architectural designs. As the sketch is being drawn, our system periodically interprets its 2.5D-geometry by identifying new junctions, edges, and faces, and then analyzing the extracted topology. The user can add detailed geometry and textures through sketches as well. This is possible through the use of databases that match partial sketches to models of detailed geometry and textures. The final product is a realistic texture-mapped 2.5D-model of the building. We show a variety of buildings that have been created using this system.