Jin Zeng

Bipartite subgraph decomposition for critically sampled wavelet filterbanks on arbitrary graphs

The observation of frequency folding in graph spectrum during down-sampling for signals on bipartite graphs-analogous to the same phenomenon in Fourier domain for regularly sampled signals-has led to the development of critically sampled wavelet filterbanks such as GraphBior. However, typical graph-signals live on general graphs that are not necessarily bipartite. To decompose a non-bipartite graph into a series of bipartite subgraphs so that two-channel filterbanks can be applied iteratively, we propose a new algorithm based on two criteria easily computed in the vertex domain aiming at compact signal representation in the wavelet domain. Given that filterbanks have minimal frequency discrimination at 1, the first criterion aims to minimize the multiplicity of mid graph frequency 1. The second criterion aims to preserve the edge structure of the original graph, which may reflect correlations among signal samples, so that a signal projected on approximated bipartite subgraphs can nonetheless be well represented using low frequency components. Experimental results show that our proposed bipartite subgraph decomposition outperforms competing proposals in terms of energy compaction.

Self-similarity-based image colorization

In this work, we tackle the problem of coloring black-and-white images, which is image colorization. Existing image colorization algorithms can be categorized into two types: scribble-based colorization algorithms and example-based colorization algorithms. Differently, we propose a hybrid scheme that combines the advantages of both categories. Given the grayscale image to be colorized and a few color scribbles (or scattered color labels) as input, the proposed method manages to colorize the grayscale image with high quality. Similar to the mechanisms in example-based colorization methods, our algorithm firstly propagates chrominance information based on the assumption that similar image patches should have similar colors. Therefore colors of some pixels can be transferred from similar patches with known colors. After that, we apply scribble-based colorization algorithm to fully colorize the grayscale image, with different confidences assigned onto the transferred color labels. Experimental results show that, the proposed method effectively utilizes the known chrominance, and provides pleasant colorizations with very few user interventions.

A tutorial on image/video coding standards

The field of image and video compression has gone through rapid growth during the past thirty years, leading to various coding standards. The main goal of continuous efforts on image/video coding standardization is to achieve low bit rate for data storage and transmission, while maintaining acceptable distortion. In this paper, various developmental stages of image and video compression standards are reviewed, including JPEG and JPEG 2000 image standards, MPEG-1, MPEG-2, MPEG-4, H.261, H.263, H.264/MPEG-4 AVC, and the latest international video standard HEVC as well as Chinese video coding standard AVS. Key features and major applications of the standards will be briefly introduced and the compression performance of the standards at each stage will be compared and discussed.

Image colorization via color propagation and rank minimization

Image colorization aims to add colors to grayscale images, which used to be a time-consuming and tedious task that requires lots of human efforts. In this paper, we present a novel colorization method based on color propagation and rank minimization. Given a small portion of chrominance values and a grayscale image, we firstly propagate the known color values to other pixels to be colorized. As the colorized image after color propagation is not accurate, we then define a confidence matrix to measure the propagation fidelity. Finally, pixels that have propagated chrominance values with confidence are colorized by rank minimization, which exploits the redundancy of natural images. Experimental results on real data set show that our proposed method achieves state-of-the-art colorization quality.

Palette-based compound image compression in HEVC by exploiting non-local spatial correlation

Non-camera captured images (also known as compound image) contain a mixture of camera-captured natural images and computer-generated graphics and texts. Nowadays, there are more and more applications calling for non-camera captured image/video compression scheme. However, current video coding standards, which are designed for natural video, treat non-camera captured video less carefully. For example, the state-of-the-art video coding standard High Efficiency Video Coding (HEVC) may blur or even remove edges in text/graphic region. A lot of schemes are proposed to preserve direction property of texts and graphics, such as palette-based intra coding. In this paper, a novel palette coding scheme is proposed for palette-based intra coding in HEVC. The palette in a block is predicted from an adaptive palette template, which records the statistical non-local spatial correlation of an image. Every block chooses its own palette using the palette template as the prediction in a rate-distortion optimized manner. Experimental results show that the proposed scheme can achieve up to 5.2% bit-rate saving compared to the state-of-the-art palette-based coding scheme in HEVC.

Bipartite Approximation for Graph Wavelet Signal Decomposition

To compactly represent a graph signal in the frequency domain, critically sampled biorthogonal wavelet filterbanks have been proposed to decompose signals on bipartite graphs. However, in practice graph signals often reside on general graph structures that are not bipartite. Thus, an original nonbipartite graph must be decomposed into a sequence of bipartite graph approximations, so that the filterbanks can be applied successively for signal decomposition. In this paper, unlike previous proposals that are heuristic in nature, we design new bipartite approximation strategies for model-based and empirically derived probability distributions. In the first case, a signal prior assumes a Gaussian Markov Random Field (GMRF) model parametrized by the original graph. In the second case, beyond the GMRF signal prior, empirical signal observations are available to compute a posterior probability distribution. In both cases, we optimize for energy compaction in the bipartite subgraphs with two criteria: the Kullback–Leibler divergence metric, which encourages preserving the spectral characteristics of the original graph; and multiplicity of eigenvalue at frequency 1 for graph Laplacian, which is the frequency with minimal energy discrimination. The relative importance between the two criteria is determined by a proposed measure that evaluates the degree of mismatch between the signal prior and posterior. Given these criteria, we first design a global numerical optimization, and then propose a local heuristic approach for fast implementation with simplifications leading to local metric computation. Experimental results show that our proposed bipartite subgraph decomposition outperforms competing proposals in terms of energy compaction.

A comprehensive study on digital image matting

Natural image matting refers to the problem of extracting the region of interest such as foreground object from an image based on user inputs like scribbles or trimap. Matting is an ill-posed problem inherently since we need to output three images out of only one input image. After a comprehensive survey and analysis of the existing matting literature, we observe that there are three key components in better estimating the alpha values, that is, the design of matting laplacian matrix, the definition of neighborhood and the choices of feature space. Based on this observation, we introduce a unified framework for digital image matting, which provides the possibility of obtaining a better understanding and direction of further improvement for image matting problem. The experimental results tested on different matting algorithms further prove the feasibility of our proposed framework.

An analytical study of subpixel-based image down-sampling patterns in frequency domain

Subpixel-based image down-sampling is a class of methods that can provide improved apparent resolution of the down-scaled image compared to the pixel-based methods. The frequency characteristics of all possible subpixel-based down-sampling patterns for RGB vertical stripes are analytically studied in this paper. Our proposed algorithm reveals that there are merely seven equivalent energy distributions in the luminance frequency spectrum. To achieve higher luminance resolution, we then calculate and choose the optimal down-sampling pattern with anti-aliasing low-pass filter designed for it so as to maximize the energy of the luminance component within the cut-off shape. Experimental results show that the proposed method provides sharper images compared to the state-of-art subpixel-based methods, with little color distortion.

Subpixel-Based Image Scaling for Grid-like Subpixel Arrangements: A Generalized Continuous-Domain Analysis Model

Subpixel-based image scaling can improve the apparent resolution of displayed images by controlling individual subpixels rather than whole pixels. However, improved luminance resolution brings chrominance distortion, making it crucial to suppress color error while maintaining sharpness. Moreover, it is challenging to develop a scheme that is applicable for various subpixel arrangements and for arbitrary scaling factors. In this paper, we address the aforementioned issues by proposing a generalized continuous-domain analysis model, which considers the low-pass nature of the human visual system (HVS). Specifically, given a discrete image and a grid-like subpixel arrangement, the signal perceived by the HVS is modeled as a 2D continuous image. Minimizing the difference between the perceived image and the continuous target image leads to the proposed scheme, which we call continuous-domain analysis for subpixel-based scaling (CASS). To eliminate the ringing artifacts caused by the ideal low-pass filtering in CASS, we propose an improved scheme, which we call CASS with Laplacian-of-Gaussian filtering. Experiments show that the proposed methods provide sharp images with negligible color fringing artifacts. Our methods are comparable with the state-of-the-art methods when applied on the RGB stripe arrangement, and outperform existing methods when applied on other subpixel arrangements.

Analysis of sampling pattern and Luma-Chroma filter design for subpixel-based image downsampling

Subpixel-based image downsampling is attractive in that it produces higher apparent resolution of down-sampled images on LCD displays. However increased luminance resolution is achieved at the price of color fringing artifacts. In this paper, we propose an algorithm to find a pleasing balance between increased resolution and color fidelity. We separate the subpixel-based downsampling into two stages, shifting followed by downsampling with anti-aliasing filtering. In stage one, we find special characteristics of the luminance and chrominance spectra of the shifted image, based on which the optimal sampling pattern is found. In stage two, anti-aliasing filters for luminance and chrominance are designed respectively. Experimental results verify that the proposed method manages to suppress color artifacts while maintaining high luminance sharpness.