Doug DeCarlo

Line drawings from 3D models

Nonphotorealistic rendering techniques, including line drawings, can be remarkably efficient at conveying shape and meaning with a minimum of visual distraction. This class will describe techniques for automated rendering of 3D models using a number of sparse line drawing styles, for both artistic and illustrative purposes. We will mathematically define lines such as silhouettes, contours, creases, suggestive contours and highlights, and apparent ridges and valleys. We then describe algorithms for finding lines efficiently, including object- and image-space methods, and discuss methods for stylization and level-of-detail control. Finally, we provide a brief introduction to concepts of visual perception, including the information content of line drawings and the effects of abstraction and detail on attention.

Pixelated image abstraction

We present an automatic method that can be used to abstract high resolution images into very low resolution outputs with reduced color palettes in the style of pixel art. Our method simultaneously solves for a mapping of features and a reduced palette needed to construct the output image. The results are an approximation to the results generated by pixel artists. We compare our method against the results of a naive process common to image manipulation programs, as well as the hand-crafted work of pixel artists. Through a formal user study and interviews with expert pixel artists we show that our results offer an improvement over the naive methods.

RigMesh: automatic rigging for part-based shape modeling and deformation

The creation of a 3D model is only the first stage of the 3D character animation pipeline. Once a model has been created, and before it can be animated, it must be rigged. Manual rigging is laborious, and automatic rigging approaches are far from real-time and do not allow for incremental updates. This is a hindrance in the real world, where the shape of a model is often revised after rigging has been performed. In this paper, we introduce algorithms and a user-interface for sketch-based 3D modeling that unify the modeling and rigging stages of the 3D character animation pipeline. Our algorithms create a rig for each sketched part in real-time, and update the rig as parts are merged or cut. As a result, users can freely pose and animate their shapes and characters while rapidly iterating on the base shape. The rigs are compatible with the state-of-the-art character animation pipeline; they consist of a low-dimensional skeleton along with skin weights identifying the surface with bones of the skeleton.

Diffusion constraints for vector graphics

The formulation of Diffusion Curves [Orzan et al. 2008] allows for the flexible creation of vector graphics images from a set of curves and colors: a diffusion process fills out the parts of the image that are away from curves. However, this model has limitations in certain situations and does not always seem to agree with how an artist wants to use the software. First, the diffusion itself cannot be controlled, only the colors. Further, the fact that color needs to be defined everywhere along the curve can lead to tedious and nonintuitive interactions. In this paper, we present a number of adaptations to diffusion curves that constrain how color is spread across the image. Specifically, we argue for the utility of controlling the speed and direction of the color diffusion, and the ability to have barriers that can be defined without the need to specify a particular color along these curves. We also describe how this can be implemented by solving a linear system, and demonstrate the effectiveness of our solution on a number of examples.

Pixelated image abstraction with integrated user constraints

We present an automatic method that can be used to abstract high resolution images into very low resolution outputs with reduced color palettes in the style of pixel art. Our method simultaneously solves for a mapping of features and a reduced palette needed to construct the output image. The results are an approximation to the results generated by pixel artists. We compare our method against the results of two naive methods common to image manipulation programs, as well as the hand-crafted work of pixel artists. Through a formal user study and interviews with expert pixel artists we show that our results offer an improvement over the naive methods. By integrating a set of manual controls into our algorithm, we give users the ability to add constraints and incorporate their own choices into the iterative process. Graphical abstractDisplay Omitted Highlights? We propose an automated image processing method that approximates pixel art. ? We compare our method to naive methods and the work of pixel artists. ? Formal user study and interviews with experts demonstrate advantages of our method. ? Additional user controls allow flexible use of feedback to guide algorithm.

Visual explanations

Human perceptual processes organize visual input to make the structure of the world explicit. Successful techniques for automatic depiction, meanwhile, create images whose structure clearly matches the visual information to be conveyed. We discuss how analyzing these structures and realizing them in formal representations can allow computer graphics to engage with perceptual science, to mutual benefit. We call these representations visual explanations: their job is to account for patterns in two dimensions as evidence of a visual world.

Depicting 3D shape using lines

Over the last few years, researchers in computer graphics have developed sophisticated mathematical descriptions of lines on 3D shapes that can be rendered convincingly as strokes in drawings. These innovations highlight fundamental questions about how human perception takes strokes in drawings as evidence of 3D structure. Answering these questions will lead to a greater scientific understanding of the flexibility and richness of human perception, as well as to practical techniques for synthesizing clearer and more compelling drawings. This paper reviews what is known about the mathematics and perception of computer-generated line drawings of shape and motivates an ongoing program of research to better characterize the shapes people see when they look at such drawings.

Binary Shading Using Appearance and Geometry

In the style of binary shading, shape and illumination are depicted using two colours, typically black and white, which form coherent lines and regions in the image. We formulate the problem of assigning colours in the rendered image as an energy minimization, computed using graph cut on the image grid. The terms of this energy come from two sources: appearance (shading) and geometry (depth and curvature). Our contributions are in the use of geometric information in determining colours, and how this information is incorporated into a graph cut approach. This optimization yields boundaries between black and white regions that tend towards being shorter and to run along geometric features like creases. We show a range of results, and demonstrate that this approach produces more coherent images than simpler approaches that make local decisions when assigning colours, or that do not use geometry.

Schema-Driven Influences in Recovering 3-D Shape from Motion in Human and Computer Vision

To investigate the role of prior knowledge in perceiving the hollow-face illusion, we set out to correlate human performance with that of a computational model that tracks facial features. Toward this goal, we prepared a video of a thin human mask, realistically painted on both the convex and concave sides, that rotated by 360 degrees. It is well known that when humans view the rotating hollow part of the mask, they perceive it as a convex face that rotates in the opposite direction. We submitted this video as input to DeCarlo & Metaxas’s (Int. J. Comput. Vis. 38(2):99–127, 2000) face tracking algorithm that uses a combination of optical flow and feature alignment to track moving faces; importantly, the algorithm uses a schema of a 3-D convex face. The algorithm was fooled, as humans are, into misperceiving a concave face as a convex face rotating in the opposite direction. Notably, this is a significant case where a computer vision algorithm “experiences” an illusion. However, when the convex-face schema was replaced with a schema that allowed both convex and concave faces, the algorithm tracked the mask accurately. The similarity in the responses of the computer vision algorithm and human observers reinforces the hypothesis that a built-in convex 3D face schema imposes a depth inversion for concave faces that in turn forces the false interpretation of motion signals.

Visualization, understanding, and design: technical perspective

analysis of exemplary hand-made work, from artists’ reflection on their practice, from psychological theories of how people understand these visualizations, and from the researchers’ own experimentation with the possibilities of technology. Realizing these design principles involves a judicious choice of visual techniques. Non-photorealism in computer graphics offers diverse ways to stylize appearances and guide the viewer’s attention. Examples include modulations of detail and weight in rendering objects, the use of cutting and transparency to depict objects in multiple layers, and even selective choices about which elements to render at all. The use here of simple line drawings, with arrows for annotation, and a constrained set of highlighted parts and exploded views, is a choice that reliably leads to clear and uncluttered imagery. To create accessible imagery with more richly varying rendering techniques, or with visualizations of additional information (forces, for example), it might be necessary to develop much more nuanced design principles. The pictures here, however, are clearly a success. It is never easy to endow computers with a deep and interesting understanding that they can share with their users. But that does not mean we should regard inference as hopeless or design as magic. As this work shows, general tools and methodologies are making it easier and easier for systems to communicate the understanding they have through clear and compelling visualizations. The results here thus take on particular significance as a benchmark in visual explanation, and a model for future systems.