Stephen N. Schiller

Depth from Defocus in the Wild

We consider the problem of two-frame depth from defocus in conditions unsuitable for existing methods yet typical of everyday photography: a handheld cellphone camera, a small aperture, a non-stationary scene and sparse surface texture. Our approach combines a global analysis of image content—3D surfaces, deformations, figure-ground relations, textures—with local estimation of joint depth-flow likelihoods in tiny patches. To enable local estimation we (1) derive novel defocus-equalization filters that induce brightness constancy across frames and (2) impose a tight upper bound on defocus blur—just three pixels in radius—through an appropriate choice of the second frame. For global analysis we use a novel piecewise-spline scene representation that can propagate depth and flow across large irregularly-shaped regions. Our experiments show that this combination preserves sharp boundaries and yields good depth and flow maps in the face of significant noise, uncertainty, non-rigidity, and data sparsity.

Interactive Vectorization

Vectorization turns photographs into vector art. Manual vectorization, where the artist traces over the image by hand, requires skill and time. On the other hand, automatic approaches allow users to generate a result by setting a few global parameters. However, global settings often leave too much detail/complexity in some parts of the image while missing important details in others. We propose interactive vectorization tools that offer more local control than automatic systems, but are more powerful and high-level than simple curve editing. Our system enables novices to vectorize images significantly faster than even experts with state-of-the-art tools.

k -curves: interpolation at local maximum curvature

We present a method for constructing almost-everywhere curvature-continuous, piecewise-quadratic curves that interpolate a list of control points and have local maxima of curvature only at the control points. Our premise is that salient features of the curve should occur only at control points to avoid the creation of features unintended by the artist. While many artists prefer to use interpolated control points, the creation of artifacts, such as loops and cusps, away from control points has limited the use of these types of curves. By enforcing the maximum curvature property, loops and cusps cannot be created unless the artist intends for them to be. To create such curves, we focus on piecewise quadratic curves, which can have only one maximum curvature point. We provide a simple, iterative optimization that creates quadratic curves, one per interior control point, that meet with G2 continuity everywhere except at inflection points of the curve where the curves are G1. Despite the nonlinear nature of curvature, our curves only obtain local maxima of the absolute value of curvature only at interpolated control points.

Advanced drawing beautification with ShipShape

Sketching is one of the simplest ways to visualize ideas. Its key advantage is its easy availability and accessibility, as it require the user to have neither deep knowledge of a particular drawing program nor any advanced drawing skills. In practice, however, all these skills become necessary to improve the visual fidelity of the resulting drawing. In this paper, we present ShipShape-a general beautification assistant that allows users to maintain the simplicity and speed of freehand sketching while still taking into account implicit geometric relations to automatically rectify the output image. In contrast to previous approaches ShipShape works with general Bezier curves, enables undo/redo operations, is scale independent, and is fully integrated into Adobe Illustrator. We show various results to demonstrate the capabilities of the proposed method. Graphical abstractDisplay Omitted HighlightsWe present a system for interactive drawing beautification.Drawings are incrementally beautified using extensible set of geometric rules.The system enables easy creation of highly regularized drawing.