Hanli Zhao

Real-time feature-aware video abstraction

This paper presents a novel feature-aware rendering system that automatically abstracts videos and images with the goal of improving the effectiveness of imagery for visual communication tasks. We integrate the bilateral grid to simplify regions of low contrast, which is faster than the separable approximation to the bilateral filter, and use a feature flow-guided anisotropic edge detection filter to enhance regions of high contrast. The edges detected in this paper are smoother, more coherent and stylistic than those of the isotropic difference-of-Gaussian filter. The presented algorithms are highly parallel, allowing a real-time performance on modern GPUs. The implementation of our approach is straightforward. Several experimental examples are given at the end of the paper to demonstrate the effectiveness of our approach.

Parallel and efficient Boolean on polygonal solids

We present a novel framework which can efficiently evaluate approximate Boolean set operations for B-rep models by highly parallel algorithms. This is achieved by taking axis-aligned surfels of Layered Depth Images (LDI) as a bridge and performing Boolean operations on the structured points. As compared with prior surfel-based approaches, this paper has much improvement. Firstly, we adopt key-data pairs to store LDI more compactly. Secondly, robust depth peeling is investigated to overcome the bottleneck of layer-complexity. Thirdly, an out-of-core tiling technique is presented to overcome the limitation of memory. Real-time feedback is provided by streaming the proposed pipeline on the many-core graphics hardware.

Real-time saliency-aware video abstraction

Existing real-time automatic video abstraction systems rely on local contrast only for identifying perceptually important information and abstract imagery by reducing contrast in low-contrast regions while artificially increasing contrast in higher contrast regions. These methods, however, may fail to accentuate an object against its background for the images with objects of low contrast over background of high contrast. To solve this problem, we propose a progressive abstraction method based on a region-of-interest function derived from an elaborate perception model. Visual contents in perceptually salient regions are emphasized, whereas the background is abstracted appropriately. In addition, the edge-preserving smoothing and line drawing algorithms in this paper are guided by a vector field which describes the flow of salient features of the input image. The whole pipeline can be executed automatically in real time on the GPU, without requiring any user intervention. Several experimental examples are shown to demonstrate the effectiveness of our approach.

Mathematical Marbling

In this paper, the proposed method takes a mathematical approach with closed-form expressions to simulate marbling. This method improves control, ease of implementation, parallelism, and speed, enabling real-time visual feedback and creation of vivid flowing animations. Users can start designs from a blank sheet, raster images, or videos.

Manifold-preserving image colorization with nonlocal estimation

In this paper, we propose a novel scribble-based colorization method which embeds colors using reconstruction weights from nonlocal nearest texture-similar neighbors in image lightness channel. Our main idea is to exploit image textural features to optimize the color distribution with an energy minimization between a pixel and K nonlocal nearest texture-similar neighbors in the feature space. A manifold structure of image lightness channel is embedded into the color channel which best preserves the geometric properties of the original space. Our method is actually the combine of global and local optimum of the cost function, which can be solved with a sparse linear system. Experimental results show that the proposed method can produce higher-quality colorizations than existing methods with sparse constraints, while having better performance in color bleeding. Moreover, simplicity and ease of implementation of our method achieves good runtime performance.

Image recoloring using geodesic distance based color harmonization

In this paper, we present a computationally simple yet effective image recoloring method based on color harmonization. Our method permits the user to obtain recolored results interactively by rotating a harmonious template after completing color harmonization. Two main improvements are made in this paper. Firstly, we give a new strategy for finding the most harmonious scheme, in terms of finding the template which best matches the hue distribution of the input image. Secondly, in order to achieve spatially coherent harmonization, geodesic distances are used to move hues lying outside the harmonious sectors to inside them. Experiments show that our approach can produce higher-quality visually pleasing recolored images than existing methods. Moreover, our method is simple and easy to implement, and has good runtime performance.

Analytical solutions for sketch-based convolution surface modeling on the GPU

Convolution surfaces are attractive for modeling objects of complex evolving topology. This paper presents some novel analytical convolution solutions for planar polygon skeletons with both finite-support and infinite-support kernel functions. We convert the double integral over a planar polygon into a simple integral along the contour of the polygon based on Green’s theorem, which reduces the computational cost and allows for efficient parallel computation on the GPU. For finite support kernel functions, a skeleton clipping algorithm is presented to compute the valid skeletons. The analytical solutions are integrated into a prototype modeling system on the GPU (Graphics Processing Unit). Our modeling system supports point, polyline and planar polygon skeletons. Complex objects with arbitrary genus can be modeled easily in an interactive way. Resulting convolution surfaces with high quality are rendered with interactive ray casting.

Semantic portrait color transfer with internet images

We present a novel color transfer method for portraits by exploring their high-level semantic information. First, a database is set up which consists of a collection of portrait images download from the Internet, and each of them is manually segmented using image matting as a preprocessing step. Second, we search the database using Face++ to find the images with similar poses to a given source portrait image, and choose one satisfactory image from the results as the target. Third, we extract portrait foregrounds from both source and target images. Then, the system extracts the semantic information, such as faces, eyes, eyebrows, lips, teeth, etc., from the extracted foreground of the source using image matting algorithms. After that, we perform color transfer between corresponding parts with the same semantic information. We get the final transferred result by seamlessly compositing different parts together using alpha blending. Experimental results show that our semantics-driven approach can generate better color transfer results for portraits than previous methods and provide users a new means to retouch their portraits.

AtelierM++: a fast and accurate marbling system

We present AtelierM++, a new interactive marbling image rendering system which allows artists to create marbling textures with real-time visual feedback on mega-pixel sized images. Marbling is a method of aqueous surface design, which can produce patterns similar to marble or other stone, hence the name. The system is based on the physical model of the traditional marbling process. We simulate real marbling by solving the Navier-Stokes equations on the graphics processing unit. We employ a third-order accurate but fast Unsplit semi-Lagragian Constrained Interpolation Profile method to reduce the numerical dissipation while retaining the stability. To simulate very sharp interface lines among different paints, a simple yet effective transformation function is applied to the paint concentrations. Several intuitive interfaces are implemented to provide flexible control for users. Extensive experimental results are shown to demonstrate both the effectiveness and efficiency of the proposed approach.

Real-Time Tone Mapping for High-Resolution HDR Images

High dynamic range rendering attempts to take an HDR image and produce a more realistic representation on a limited range computer monitor. Although several tone mapping operators have been proposed in recent years, no evaluation has yet been undertaken to explore which operator is more suitable for hardware implementation. In this paper, we begin with our novel GPU implementations of two state-of-the-art operators in real time. Then several experimental results using eight GPU-based tone mapping operators are presented to evaluate which one is better with regard to running efficiency. Our GPU implementation of the Pattanaik operator can achieve real-time performance even on high-resolution HDR images. In addition, we believe that many real-time applications, including HDR video player and environment mapping with HDR textures in games, will benefit from our novel approach.