Computer Science

A haploscope is an optical system which produces a carefully controlled virtual image. Since the development of Wheatstone's original stereoscope in 1838, haploscopes have been used to measure perceptual properties of human stereoscopic vision. This paper presents an augmented reality (AR) haploscope, which allows the viewing of virtual objects superimposed against the real world. Our lab has used generations of this device to make a careful series of perceptual measurements of AR phenomena, which have been described in publications over the previous 8 years. This paper systematically describes the design, assembly, calibration, and measurement of our AR haploscope. These methods have been developed and improved in our lab over the past 10 years. Despite the fact that 180 years have elapsed since the original report of Wheatstone's stereoscope, we have not previously found a paper that describes these kinds of details.

An intuitive design method is proposed for generating developable ruled B-spline surfaces from a sequence of straight line segments indicating the surface shape. The first and last line segments are enforced to be the head and tail ruling lines of the resulting surface while the interior lines are required to approximate rulings on the resulting surface as much as possible. This manner of developable surface design is conceptually similar to the popular way of the freeform curve and surface design in the CAD community, observing that a developable ruled surface is a single parameter family of straight lines. This new design mode of the developable surface also provides more flexibility than the widely employed way of developable surface design from two boundary curves of the surface. The problem is treated by numerical optimization methods with which a particular level of distance error is allowed. We thus provide an effective tool for creating surfaces with a high degree of developability when the input control rulings do not lie in exact developable surfaces. We consider this ability as the superiority over analytical methods in that it can deal with arbitrary design inputs and find practically useful results.

In this paper we construct developable surface patches which are bounded by two rational or NURBS curves, though the resulting patch is not a rational or NURBS surface in general. This is accomplished by reparameterizing one of the boundary curves. The reparameterization function is the solution of an algebraic equation. For the relevant case of cubic or cubic spline curves, this equation is quartic at most, quadratic if the curves are Bezier or splines and lie on parallel planes, and hence it may be solved either by standard analytical or numerical methods.

Deformable shape modeling approaches that describe objects in terms of their medial axis geometry (e.g., m-reps [Pizer et al., 2003]) yield rich geometrical features that can be useful for analyzing the shape of sheet-like biological structures, such as the myocardium. We present a novel shape analysis approach that combines the benefits of medial shape modeling and diffeomorphometry. Our algorithm is formulated as a problem of matching shapes using diffeomorphic flows under constraints that approximately preserve medial axis geometry during deformation. As the result, correspondence between the medial axes of similar shapes is maintained. The approach is evaluated in the context of modeling the shape of the left ventricular wall from 3D echocardiography images.

Capturing the 3D geometry of transparent objects is a challenging task, ill-suited for general-purpose scanning and reconstruction techniques, since these cannot handle specular light transport phenomena. Existing state-of-the-art methods, designed specifically for this task, either involve a complex setup to reconstruct complete refractive ray paths, or leverage a data-driven approach based on synthetic training data. In either case, the reconstructed 3D models suffer from over-smoothing and loss of fine detail. This paper introduces a novel, high precision, 3D acquisition and reconstruction method for solid transparent objects. Using a static background with a coded pattern, we establish a mapping between the camera view rays and locations on the background. Differentiable tracing of refractive ray paths is then used to directly optimize a 3D mesh approximation of the object, while simultaneously ensuring silhouette consistency and smoothness. Extensive experiments and comparisons demonstrate the superior accuracy of our method.

We propose Differentiable Surface Splatting (DSS), a high-fidelity differentiable renderer for point clouds. Gradients for point locations and normals are carefully designed to handle discontinuities of the rendering function. Regularization terms are introduced to ensure uniform distribution of the points on the underlying surface. We demonstrate applications of DSS to inverse rendering for geometry synthesis and denoising, where large scale topological changes, as well as small scale detail modifications, are accurately and robustly handled without requiring explicit connectivity, outperforming state-of-the-art techniques. The data and code are at this https URL.

Derivatives of computer graphics, image processing, and deep learning algorithms have tremendous use in guiding parameter space searches, or solving inverse problems. As the algorithms become more sophisticated, we no longer only need to differentiate simple mathematical functions, but have to deal with general programs which encode complex transformations of data. This dissertation introduces three tools for addressing the challenges that arise when obtaining and applying the derivatives for complex graphics algorithms. Traditionally, practitioners have been constrained to composing programs with a limited set of operators, or hand-deriving derivatives. We extend the image processing language Halide with reverse-mode automatic differentiation, and the ability to automatically optimize the gradient computations. This enables automatic generation of the gradients of arbitrary Halide programs, at high performance, with little programmer effort. In 3D rendering, the gradient is required with respect to variables such as camera parameters, geometry, and appearance. However, computing the gradient is challenging because the rendering integral includes visibility terms that are not differentiable. We introduce, to our knowledge, the first general-purpose differentiable ray tracer that solves the full rendering equation, while correctly taking the geometric discontinuities into account. Finally, we demonstrate that the derivatives of light path throughput can also be useful for guiding sampling in forward rendering. Simulating light transport in the presence of multi-bounce glossy effects and motion in 3D rendering is challenging due to the hard-to-sample high-contribution areas. We present a Markov Chain Monte Carlo rendering algorithm that extends Metropolis Light Transport by automatically and explicitly adapting to the local integrand, thereby increasing sampling efficiency.

We present Diffusion Structures, a family of resilient shell structures from the eigenfunctions of a pair of novel diffusion operators. This approach is based on Michell's theorem but avoids expensive non-linear optimization with computation that amounts to constructing and solving two generalized eigenvalue problems to generate two sets of stripe patterns. This structure family can be generated quickly, and navigated in real-time using a small number of tuneable parameters.

New algorithmic and hardware developments over the past two decades have enabled interactive ray tracing of small to modest sized scenes, and are finding growing popularity in scientific visualization and games. However, interactive ray tracing has not been as widely explored in the context of production film rendering, where challenges due to the complexity of the models and, from a practical standpoint, their unavailability to the wider research community, have posed significant challenges. The recent release of the Disney Moana Island Scene has made one such model available to the community for experimentation. In this paper, we detail the challenges posed by this scene to an interactive ray tracer, and the solutions we have employed and developed to enable interactive path tracing of the scene with full geometric and shading detail, with the goal of providing insight and guidance to other researchers.

We propose visual creations that put differences in algorithms and humans \emph{perceptions} into perspective. We exploit saliency maps of neural networks and visual focus of humans to create diptychs that are reinterpretations of an original image according to both machine and human attentions. Using those diptychs as a qualitative evaluation of perception, we discuss some crucial issues of current \textit{task-oriented} artificial intelligence.