Zhuo Su

Corruptive Artifacts Suppression for Example-Based Color Transfer

Example-based color transfer is a critical operation in image editing but easily suffers from some corruptive artifacts in the mapping process. In this paper, we propose a novel unified color transfer framework with corruptive artifacts suppression, which performs iterative probabilistic color mapping with self-learning filtering scheme and multiscale detail manipulation scheme in minimizing the normalized Kullback-Leibler distance. First, an iterative probabilistic color mapping is applied to construct the mapping relationship between the reference and target images. Then, a self-learning filtering scheme is applied into the transfer process to prevent from artifacts and extract details. The transferred output and the extracted multi-levels details are integrated by the measurement minimization to yield the final result. Our framework achieves a sound grain suppression, color fidelity and detail appearance seamlessly. For demonstration, a series of objective and subjective measurements are used to evaluate the quality in color transfer. Finally, a few extended applications are implemented to show the applicability of this framework.

A novel image decomposition approach and its applications

The current state-of-the-art edge-preserving decomposition techniques may not be able to fully separate textures while preserving edges. This may generate artifacts in some applications, e.g., edge detection, texture transfer, etc. To solve this problem, a novel image decomposition approach based on explicit texture separation from large scale components of an image is presented. We first apply a Gaussian structure-texture decomposition, to separate the majority of textures out of the input image. However, residual textures are still visible around the strong edges. To remove these residuals, an asymmetric sampling operator is proposed and followed by a joint bilateral correction to remove an excessive blur effect. We demonstrate that our approach is well suited for the tasks such as texture transfer, edge detection, non-photorealistic rendering, and tone mapping. The results show our approach outperforms existing state-of-the-art image decomposition approaches.

Robust tracking via discriminative sparse feature selection

In this paper, we propose a novel generative tracking approach based on discriminative sparse feature selection. The sparse features are the discriminative sparse representation of samples, which are achieved by learning a compact and discriminative dictionary. Besides the target templates, the proposed approach also incorporates the close-background templates to approximate the partial variations. We learn the dictionary and a classifier together, and search the tracking result with the maximum similarity and the minimal reconstruction error criterion using the discrimination of sparse features. In addition, we resample the close-background templates and update the dictionary in an adaptive way during tracking. Experimental results on several challenging video sequences demonstrate that the proposed approach has more favorable performance than the state-of-the-art approaches.

Mesh-based anisotropic cloth deformation for virtual fitting

According to the anisotropic property in most real-world cloth for virtual fitting, this paper proposes a novel dynamic cloth simulation method via geometric deformation energy model that preserves geometric features well to achieve cloth behaviors with various material effects. We first construct an objective deformation energy with the terms including vertex position, edge length, dihedral angle, and gravitation, then we conduct a numerical solution in the least square sense. In order to establish the dynamic cloth deformation solution, we further analyze the corresponding relationship between different weights in front of geometric energy terms and material properties by comparison with the real photographs of typical real fabrics. Establishing a dynamic weight-regulation measure can model similar cloth anisotropic behaviors for virtual fitting applications in digital home. The experiments show that our approach effectively provide more rich cloth deformation results with distinctive material effects.

Example-Based Image Colorization Using Locality Consistent Sparse Representation

Image colorization aims to produce a natural looking color image from a given gray-scale image, which remains a challenging problem. In this paper, we propose a novel example-based image colorization method exploiting a new locality consistent sparse representation. Given a single reference color image, our method automatically colorizes the target gray-scale image by sparse pursuit. For efficiency and robustness, our method operates at the superpixel level. We extract low-level intensity features, mid-level texture features, and high-level semantic features for each superpixel, which are then concatenated to form its descriptor. The collection of feature vectors for all the superpixels from the reference image composes the dictionary. We formulate colorization of target superpixels as a dictionary-based sparse reconstruction problem. Inspired by the observation that superpixels with similar spatial location and/or feature representation are likely to match spatially close regions from the reference image, we further introduce a locality promoting regularization term into the energy formulation, which substantially improves the matching consistency and subsequent colorization results. Target superpixels are colorized based on the chrominance information from the dominant reference superpixels. Finally, to further improve coherence while preserving sharpness, we develop a new edge-preserving filter for chrominance channels with the guidance from the target gray-scale image. To the best of our knowledge, this is the first work on sparse pursuit image colorization from single reference images. Experimental results demonstrate that our colorization method outperforms the state-of-the-art methods, both visually and quantitatively using a user study.

Deep patch-wise colorization model for grayscale images

To handle the colorization problem, we propose a deep patch-wise colorization model for grayscale images. Distinguished with some constructive color mapping models with complicated mathematical priors, we alternately apply two loss metric functions in the deep model to suppress the training errors under the convolutional neural network. To address the potential boundary artifacts, a refinement scheme is presented inspired by guided filtering. In the experiment section, we summarize our network parameters setting in practice, including the patch size, amount of layers and the convolution kernels. Our experiments demonstrate this model can output more satisfactory visual colorizations compared with the state-of-the-art methods. Moreover, we prove our method has extensive application domains and can be applied to stylistic colorization.

Technical Section: Material-aware cloth simulation via constrained geometric deformation

Most real-world cloth consists of nonlinear material and exhibits anisotropic behavior. This paper proposes an efficient and expressive mesh deformation method to obtain realistic cloth shapes with various cloth materials. The key idea in this work is to model the cloth using a mesh-based deformation energy that is composed of several energy terms and to fit the weighting coefficients of the terms from real data. We first develop a direct geometrical material measurement method for testing the recovery, stretching and bending behaviors of different real cloth samples. Then, we separate the geometric deformation energy into three terms related to the vertex position, edge length and bending of the dihedral angle, respectively, and the weights for the three energy terms are learned from the data measured with real cloth. Reusing the weights for the geometric deformation by a numerical solution in the least square sense can model similar cloth behavior. The experiments show that our method effectively provides rich cloth simulation results that are able to capture distinctive material effects.

ℒ0 Gradient-Preserving Color Transfer

This paper presents a new two‐step color transfer method which includes color mapping and detail preservation. To map source colors to target colors, which are from an image or palette, the proposed similarity‐preserving color mapping algorithm uses the similarities between pixel color and dominant colors as existing algorithms and emphasizes the similarities between source image pixel colors. Detail preservation is performed by an ℒ0 gradient‐preserving algorithm. It relaxes the large gradients of the sparse pixels along color region boundaries and preserves the small gradients of pixels within color regions. The proposed method preserves source image color similarity and image details well. Extensive experiments demonstrate that the proposed approach has achieved a state‐of‐art visual performance.

A 3D model perceptual feature metric based on global height field

Human visual attention system tends to be attracted to perceptual feature points on 3D model surfaces. However, purely geometric-based feature metrics may be insufficient to extract perceptual features, because they tend to detect local structure details. Intuitively, the perceptual importance degree of vertex is associated with the height of its geometry position between original model and a datum plane. So, we propose a novel and straightforward method to extract perceptually important points based on global height field. Firstly, we construct spectral domain using Laplace–Beltrami operator, and we perform spectral synthesis to reconstruct a rough approximation of the original model by adopting low-frequency coefficients, and make it as the 3D datum plane. Then, to build global height field, we calculate the Euclidean distance between vertex geometry position on original surface and the one on 3D datum plane. Finally, we set a threshold to extract perceptual feature vertices. We implement our technique on several 3D mesh models and compare our algorithm to six state-of-the-art interest points detection approaches. Experimental results demonstrate that our algorithm can accurately capture perceptually important points on arbitrary topology 3D model.

A data-driven editing framework for automatic 3D garment modeling

Exploring shape variations on virtual garments is significant but challenging to the aspect of 3D garment modeling. In this paper, we propose a data-driven editing framework for automatic 3D garment modeling, which includes semantic garment segmentation, probabilistic reasoning for component suggestion, and garment component merging. The key idea in this work is to develop a simple but effective garment synthesis that utilizes a continuous style description, which can be characterized by the ratio of area and boundary length on garment components. First, a semi-supervised learning algorithm is proposed to simultaneously segment and label the components in 3D garments. Second, a set of matchable probability measurement is applied to recommend components that can be regarded as a new 3D garment. Third, a variation synthesis is developed to satisfy the garment style criteria while ensuring the realistic-looking plausibility of the results. As demonstrated by the experiments, our method is able to generate various reasonable garments with material effects to enrich existing 3D garments.