Zhiwei Xiong

Robust Web Image/Video Super-Resolution

This paper proposes a robust single-image super-resolution method for enlarging low quality web image/video degraded by downsampling and compression. To simultaneously improve the resolution and perceptual quality of such web image/video, we bring forward a practical solution which combines adaptive regularization and learning-based super-resolution. The contribution of this work is twofold. First, we propose to analyze the image energy change characteristics during the iterative regularization process, i.e., the energy change ratio between primitive (e.g., edges, ridges and corners) and nonprimitive fields. Based on the revealed convergence property of the energy change ratio, appropriate regularization strength can then be determined to well balance compression artifacts removal and primitive components preservation. Second, we verify that this adaptive regularization can steadily and greatly improve the pair matching accuracy in learning-based super-resolution. Consequently, their combination effectively eliminates the quantization noise and meanwhile faithfully compensates the missing high-frequency details, yielding robust super-resolution performance in the compression scenario. Experimental results demonstrate that our solution produces visually pleasing enlargements for various web images/videos.

Block-Based Image Compression With Parameter-Assistant Inpainting

This correspondence presents an image compression approach that integrates our proposed parameter-assistant inpainting (PAI) to exploit visual redundancy in color images. In this scheme, we study different distributions of image regions and represent them with a model class. Based on that, an input image at the encoder side is divided into featured and non-featured regions at block level. The featured blocks fitting the predefined model class are coded by a few parameters, whereas the non-featured blocks are coded traditionally. At the decoder side, the featured regions are restored through PAI relying on both delivered parameters and surrounding information. Experimental results show that our method outperforms JPEG in featured regions by an average bit-rate saving of 76% at similar perceptual quality levels.

Real-Time Scalable Depth Sensing With Hybrid Structured Light Illumination

Time multiplexing (TM) and spatial neighborhood (SN) are two mainstream structured light techniques widely used for depth sensing. The former is well known for its high accuracy and the latter for its low delay. In this paper, we explore a new paradigm of scalable depth sensing to integrate the advantages of both the TM and SN methods. Our contribution is twofold. First, we design a set of hybrid structured light patterns composed of phase-shifted fringe and pseudo-random speckle. Under the illumination of the hybrid patterns, depth can be decently reconstructed either from a few consecutive frames with the TM principle for static scenes or from a single frame with the SN principle for dynamic scenes. Second, we propose a scene-adaptive depth sensing framework based on which a global or region-wise optimal depth map can be generated through motion detection. To validate the proposed scalable paradigm, we develop a real-time (20 fps) depth sensing system. Experimental results demonstrate that our method achieves an efficient balance between accuracy and speed during depth sensing that has rarely been exploited before.

Unambiguous 3D measurement from speckle-embedded fringe

This paper proposes a novel phase-shifting method for fast, accurate, and unambiguous 3D shape measurement. The basic idea is embedding a speckle-like signal in three sinusoidal fringe patterns to eliminate the phase ambiguity, but without reducing the fringe amplitude or frequency. The absolute depth is then recovered through a robust region-wise voting strategy relying on the embedded signal. Using the theoretical minimum of only three images, the proposed method greatly facilitates the application of phase shifting in time-critical conditions. Moreover, the proposed method is resistant to the global illumination effects, as the fringe patterns used are with a single high frequency. Based on the proposed method, we further demonstrate a real-time, high-precision 3D scanning system with an off-the-shelf projector and a commodity camera.

Depth Acquisition from Density Modulated Binary Patterns

This paper proposes novel density modulated binary patterns for depth acquisition. Similar to Kinect, the illumination patterns do not need a projector for generation and can be emitted by infrared lasers and diffraction gratings. Our key idea is to use the density of light spots in the patterns to carry phase information. Two technical problems are addressed here. First, we propose an algorithm to design the patterns to carry more phase information without compromising the depth reconstruction from a single captured image as with Kinect. Second, since the carried phase is not strictly sinusoidal, the depth reconstructed from the phase contains a systematic error. We further propose a pixel-based phase matching algorithm to reduce the error. Experimental results show that the depth quality can be greatly improved using the phase carried by the density of light spots. Furthermore, our scheme can achieve 20 fps depth reconstruction with GPU assistance.

Image hallucination with feature enhancement

Example-based super-resolution recovers missing high frequencies in a magnified image by learning the correspondence between co-occurrence examples at two different resolution levels. As high-resolution examples usually contain more details and are of higher dimensionality in comparison with low-resolution ones, the mapping from low-resolution to high-resolution is an ill-posed problem. Rather than imposing more complicated mapping constraints, we propose to improve the mapping accuracy by enhancing low-resolution examples in terms of mapped features, e.g., derivatives and primitives. A feature enhancement method is presented through a combination of interpolation with prefiltering and non-blind sparse prior deblurring. By enhancing low-resolution examples, unique feature information carried by high-resolution examples is decreased. This regularization reduces the intrinsic dimensionality disparity between two different resolution examples and thus improves the feature mapping accuracy. Experiments demonstrate our super-resolution scheme with feature enhancement produces high quality results both perceptually and quantitatively.

Pseudo-sequence-based light field image compression

We propose a pseudo-sequence-based scheme for light field image compression. In our scheme, the raw image captured by a light field camera is decomposed into multiple views according to the lenslet array of that camera. These views constitute a pseudo sequence like video, and the redundancy between views is exploited by a video encoder. The specific coding order of views, prediction structure, and rate allocation have been investigated for encoding the pseudo sequence. Experimental results show the superior performance of our scheme, which achieves as high as 6.6 dB gain compared with directly encoding the raw image by the legacy JPEG.

Accurate Dynamic 3D Sensing With Fourier-Assisted Phase Shifting

Phase shifting profilometry (PSP) and Fourier transform profilometry (FTP) are two well-known fringe analysis methods for 3D sensing. PSP offers high accuracy but requires multiple images; FTP uses a single image but is limited in its accuracy. In this paper, we propose a novel Fourier-assisted phase shifting (FAPS) method for accurate dynamic 3D sensing. Our key observation is that the motion vulnerability of multi-shot PSP can be overcome through single-shot FTP, while the high accuracy of PSP is preserved. Moreover, to solve the phase ambiguity of complex scenes without additional images, we propose an efficient parallel spatial unwrapping strategy that embeds a sparse set of markers in the fringe patterns. Our dynamic 3D sensing system based on the above principles demonstrates superior performance over previous structured light techniques, including PSP, FTP, and Kinect.

Example-Based Super-Resolution With Soft Information and Decision

The one-to-one correspondence between co-occurrence image patches of two different resolutions is extensively used in example-based super-resolution (SR). Due to the dimensionality gap between low resolution (LR) and high resolution (HR) spaces, however, an LR patch may correspond to a number of HR patches in practice. This ambiguity is difficult to be overcome with examples representing a deterministic mapping. In this paper, we propose a statistical method for exploiting the one-to-many correspondence between LR and HR patches, which we call soft information and decision. Soft information means an LR patch is mapped to a pixel-wise distribution of all its possible HR counterparts, rather than a single or a limited set of HR candidates. Relying on the soft information, example-based SR is then regarded as an optimization problem to best preserve the local consistency in the recovered HR image. This problem is solved with an efficient message passing algorithm with a factor graph model. The final decision on the HR pixel value is made upon the maximum a posteriori estimation and is called a soft decision. Experimental results demonstrate the superiority of the proposed method compared with the state-of-the-art methods, in terms of both the subjective and objective quality of synthesized HR images.

Dual-camera design for coded aperture snapshot spectral imaging

Coded aperture snapshot spectral imaging (CASSI) provides an efficient mechanism for recovering 3D spectral data from a single 2D measurement. However, since the reconstruction problem is severely underdetermined, the quality of recovered spectral data is usually limited. In this paper we propose a novel dual-camera design to improve the performance of CASSI while maintaining its snapshot advantage. Specifically, a beam splitter is placed in front of the objective lens of CASSI, which allows the same scene to be simultaneously captured by a grayscale camera. This uncoded grayscale measurement, in conjunction with the coded CASSI measurement, greatly eases the reconstruction problem and yields high-quality 3D spectral data. Both simulation and experimental results demonstrate the effectiveness of the proposed method.