Jan Eric Kyprianidis

State of the "Art”: A Taxonomy of Artistic Stylization Techniques for Images and Video

This paper surveys the field of nonphotorealistic rendering (NPR), focusing on techniques for transforming 2D input (images and video) into artistically stylized renderings. We first present a taxonomy of the 2D NPR algorithms developed over the past two decades, structured according to the design characteristics and behavior of each technique. We then describe a chronology of development from the semiautomatic paint systems of the early nineties, through to the automated painterly rendering systems of the late nineties driven by image gradient analysis. Two complementary trends in the NPR literature are then addressed, with reference to our taxonomy. First, the fusion of higher level computer vision and NPR, illustrating the trends toward scene analysis to drive artistic abstraction and diversity of style. Second, the evolution of local processing approaches toward edge-aware filtering for real-time stylization of images and video. The survey then concludes with a discussion of open challenges for 2D NPR identified in recent NPR symposia, including topics such as user and aesthetic evaluation.

Image and Video Abstraction by Anisotropic Kuwahara Filtering

We present a non‐photorealistic rendering technique to transform color images and videos into painterly abstractions. It is based on a generalization of the Kuwahara filter that is adapted to the local shape of features, derived from the smoothed structure tensor. Contrary to conventional edge‐preserving filters, our filter generates a painting‐like flattening effect along the local feature directions while preserving shape boundaries. As opposed to conventional painting algorithms, it produces temporally coherent video abstraction without extra processing. The GPU implementation of our method processes video in real‐time. The results have the clearness of cartoon illustrations but also exhibit directional information as found in oil paintings.

Image and Video Abstraction by Coherence‐Enhancing Filtering

In this work, we present a non‐photorealistic rendering technique to create stylized abstractions from color images and videos. Our approach is based on adaptive line integral convolution in combination with directional shock filtering. The smoothing process regularizes directional image features while the shock filter provides a sharpening effect. Both operations are guided by a flow field derived from the structure tensor. To obtain a high‐quality flow field, we present a novel smoothing scheme for the structure tensor based on Poissons equation. Our approach effectively regularizes anisotropic image regions while preserving the overall image structure and achieving a consistent level of abstraction. Moreover, it is suitable for per‐frame filtering of video and can be efficiently implemented to process content in real‐time.

Image Abstraction by Structure Adaptive Filtering

In this work we present a framework of automatic non-photorealistic image processing techniques that create simplified stylistic illustrations from color images, videos and 3D renderings. To smooth low-contrast regions while preserving edges, we present a new fast separated implementation of the bilateral filter. Our approach works by filtering in direction of the gradient and then filtering the intermediate result in perpendicular direction. When applied iteratively, our approach does not suffer from horizontal or vertical artifacts and creates smooth output at curved boundaries. To extract salient important edges we first apply a one-dimensional difference-of-Gaussians filter in direction of the gradient and then apply smoothing along a flow field which we derive from the smoothed structure tensor. Our method creates smooth coherent output for line and curve segments.

Special Section on CANS: XDoG: An eXtended difference-of-Gaussians compendium including advanced image stylization

Recent extensions to the standard difference-of-Gaussians (DoG) edge detection operator have rendered it less susceptible to noise and increased its aesthetic appeal. Despite these advances, the technical subtleties and stylistic potential of the DoG operator are often overlooked. This paper offers a detailed review of the DoG operator and its extensions, highlighting useful relationships to other image processing techniques. It also presents many new results spanning a variety of styles, including pencil-shading, pastel, hatching, and woodcut. Additionally, we demonstrate a range of subtle artistic effects, such as ghosting, speed-lines, negative edges, indication, and abstraction, all of which are obtained using an extended DoG formulation, or slight modifications thereof. In all cases, the visual quality achieved by the extended DoG operator is comparable to or better than those of systems dedicated to a single style.

Interactive Visualization of Generalized Virtual 3D City Models using Level-of-Abstraction Transitions

Virtual 3D city models play an important role in the communication of complex geospatial information in a growing number of applications, such as urban planning, navigation, tourist information, and disaster management. In general, homogeneous graphic styles are used for visualization. For instance, photorealism is suitable for detailed presentations, and non‐photorealism or abstract stylization is used to facilitate guidance of a viewers gaze to prioritized information. However, to adapt visualization to different contexts and contents and to support saliency‐guided visualization based on user interaction or dynamically changing thematic information, a combination of different graphic styles is necessary. Design and implementation of such combined graphic styles pose a number of challenges, specifically from the perspective of real‐time 3D visualization. In this paper, the authors present a concept and an implementation of a system that enables different presentation styles, their seamless integration within a single view, and parametrized transitions between them, which are defined according to tasks, camera view, and image resolution. The paper outlines potential usage scenarios and application fields together with a performance evaluation of the implementation.

Image stylization by interactive oil paint filtering

This paper presents an interactive system for transforming images into an oil paint look. The system comprises two major stages. First, it derives dominant colors from an input image for feature-aware recolorization and quantization to conform with a global color palette. Afterwards, it employs non-linear filtering based on the smoothed structure adapted to the main feature contours of the quantized image to synthesize a paint texture in real-time. Our filtering approach leads to homogeneous outputs in the color domain and enables creative control over the visual output, such as color adjustments and per-pixel parametrizations by means of interactive painting. To this end, our system introduces a generalized brush-based painting interface that operates within parameter spaces to locally adjust the level of abstraction of the filtering effects. Several results demonstrate the various applications of our filtering approach to different genres of photography. Graphical abstractDisplay Omitted HighlightsAn approach for image stylization with an oil paint look by means of image filtering is proposed.A method for image quantization is introduced that is based on the dominant colors derived from local image regions.A real-time paint texture synthesis is proposed based on the smoothed image structure.An interactive painting system is provided that supports brush-based per-pixel parametrizations of local image filters.

Out‐of‐Core GPU‐based Change Detection in Massive 3D Point Clouds

If sites, cities, and landscapes are captured at different points in time using technology such as LiDAR, large collections of 3D point clouds result. Their efficient storage, processing, analysis, and presentation constitute a challenging task because of limited computation, memory, and time resources. In this work, we present an approach to detect changes in massive 3D point clouds based on an out-of-core spatial data structure that is designed to store data acquired at different points in time and to efficiently attribute 3D points with distance information. Based on this data structure, we present and evaluate different processing schemes optimized for performing the calculation on the CPU and GPU. In addition, we present a point-based rendering technique adapted for attributed 3D point clouds, to enable effective out-of-core real-time visualization of the computation results. Our approach enables conclusions to be drawn about temporal changes in large highly accurate 3D geodata sets of a captured area at reasonable preprocessing and rendering times. We evaluate our approach with two data sets from different points in time for the urban area of a city, describe its characteristics, and report on applications.

Real-time rendering of water surfaces with cartography-oriented design

More than 70% of the Earths surface is covered by oceans, seas, and lakes, making water surfaces one of the primary elements in geospatial visualization. Traditional approaches in computer graphics simulate and animate water surfaces in the most realistic ways. However, to improve orientation, navigation, and analysis tasks within 3D virtual environments, these surfaces need to be carefully designed to enhance shape perception and land-water distinction. We present an interactive system that renders water surfaces with cartography-oriented design using the conventions of mapmakers. Our approach is based on the observation that hand-drawn maps utilize and align texture features to shorelines with non-linear distance to improve figure-ground perception and express motion. To obtain local orientation and principal curvature directions, first, our system computes distance and feature-aligned distance maps. Given these maps, waterlining, water stippling, contour-hatching, and labeling are applied in real-time with spatial and temporal coherence. The presented methods can be useful for map exploration, landscaping, urban planning, and disaster management, which is demonstrated by various real-world virtual 3D city and landscape models.

Approaches to Image Abstraction for Photorealistic Depictions of Virtual 3D Models

In our contribution, we present approaches of automatic image abstraction, applied to images and image sequences derived as views of virtual 3D city models and landscape models. We first discuss the requirements of cartography-specific visualization based on the virtual globe metaphor as well as the specific characteristics and deficiencies of visualization based on photorealism. We introduce a concept that extends the classical visualization pipeline by cartography-specific functionality, object-space and image-space abstraction, which also represent the two principle ways for implementing cartographic visualization systems. Abstraction provides the prerequisites to visually communicate uncertainty, to simplify and filter detailed elements, and to clearly encode displayed information of complex geospatial information. In addition, it offers many degrees of freedom for artistic and stylistic design of cartographic products. Furthermore, we outline general working principles and implementation of an automatic image-space abstraction technique we developed that creates high-quality, simplified, stylistic illustrations from color images, videos, and 3D renderings.