Minmin Shen

Down-Sampling Based Video Coding Using Super-Resolution Technique

It has been reported that oversampling a still image before compression does not guarantee a good image quality. Similarly, down-sampling before video compression in low bit rate video coding may alleviate the blocking effect and improve peak signal-to-noise ratio of the decoded frames. When the number of discrete cosine transform coefficients is reduced in such a down-sampling based coding (DBC), the bit budget of each coefficient will increase, thus reduce the quantization error. A DBC video coding scheme is proposed in this paper, where a super-resolution technique is employed to restore the down-sampled frames to their original resolutions. The performance improvement of the proposed DBC scheme is analyzed at low bit rates, and verified by experiments.

A fast algorithm for rain detection and removal from videos

Detection and removal of rain is important in outdoor surveillance vision systems, since the appearance of rain strikes degrades the performance of various vision-based applications. The existing algorithms address the issue of detecting rain only in the irradiance light field, thus require dozens of successive frames to compute the temporal correlation of rain. Combining the properties of rain in irradiance light field and motion field, this paper presents a new approach for rain detection and removal using only three successive frames. In this approach, motion data are used to differentiate rain from other moving objects. A smoothing method based on anisotropic diffusion is proposed for rain removal. Experimental results verify the efficacy of our algorithm.

Down-sampling based video coding with super-resolution technique

It has been proven that performance of video coding at low bit rates can be improved by down-sampling a video before compression and up-sampling it after decompression. The down-sampling based coding (DBC) provides more bit budget for DCT coefficients, which preserve more high order coefficients. In this paper we propose a new DBC which uses a modified example-based super-resolution (SR) algorithm to restore the original resolution of down-sampled video signals. Experiments have shown the proposed scheme achieve PSNR improvement and better visual quality at low bit rates as compared with the H.264/AVC video coding, as well as the DBC using a general locally linear embedding (LLE) method as in [1].

A novel scalable video coding scheme using super resolution techniques

A novel scheme of scalable video coding (SVC) using super-resolution techniques is proposed in this paper. Utilizing the spatial/temporal scalability of H.264/AVC, we encode half of input high-resolution(HR) frames and their low-resolution (LR) counterparts in a video sequence and employ super-resolution (SR) method to reconstruct skipped frames during decoding. The payload saved from skipped frames can be used to improve the quality of encoded frames. This scheme provides a choice of SVC to improve the quality of HR frames while maintaining low bit rate transmission and reducing encoding complexity. Experiments show that our scheme performs well in both peak signal to noise ratio (PSNR) and subjective visual quality at low bit rate.

Performance of reconstruction-based super-resolution with regularization

From the perspective of linear algebra, the performance of super-resolution reconstruction (SR) depends on the conditioning of the linear system characterizing the degradation model. This is analyzed in the Fourier domain using the perturbation theory. By proposing a new SR error bound in terms of the point spread function (PSF), we reveal that the blur function dominates the condition number (CN) of degradation matrix, and the advantage of non-integer magnification factors (MFs) over the integer ones comes from sampling zero crossings of the DFT of the PSF. We also explore the effect of regularization by integrating it into the SR model, and investigate the influence of the optimal regularization parameter. A tighter error bound is derived given the optimal regularization parameter. Two curves of error bounds vs. MFs are presented, and verified by processing real images. It explains that with proper regularization, SR at the integer MFs is still valid.

Super-resolution from observations with variable zooming ratios

Super-resolution reconstruction (SR) is a technique for estimating a high resolution (HR) image from multiple low resolution (LR) copies captured from the same scene. Most of the existing SR algorithms are based on the assumption that the scene moves parallel to the camera lens with translational or rotational motion. However, such an assumption may not be held if zooming exists when acquiring LR images. We present in this paper a new linear model to represent the relationship between the HR image and the LR images captured with arbitrary sampling lattices. Based on this model, a MAP based SR algorithm is proposed. Experimental results verify the improvements on the visual quality of our framework

Rain streak removal by multi-frame-based anisotropic filtering

Dynamic weather conditions, such as rain and snow, often produce strong intensity discontinuity among frames, thus seriously degrade their visual or compression performance. How to remove these artifacts is a challenging task and has been intensively studies recently. The state-of-the-art algorithms detect these scratches before removing them from the scene. Visual effect of rain or snow is complex and difficult to be distinguished from the background; hence the precision of its detection and segmentation by hard decision is usually unsatisfactory. As an anisotropic filter performs well in structural noise removal, such as linear, planar as well as isotropic noise, it is utilized in this paper to analyze image content and suppress scratch noise simultaneously. Compared with the state-of-the-art algorithms, the proposed algorithm is better and more robust in dynamic scenes.

A novel dementia diagnosis strategy on arterial spin labeling magnetic resonance images via pixel-wise partial volume correction and ranking

Arterial Spin Labeling (ASL) is an emerging magnetic resonance imaging technique attracting increasing attention in dementia diagnosis only beginning from recent years. ASL is capable to provide direct and quantitative measurement of cerebral blood flow (CBF) of scanned patients, so that brain atrophy of demented patients could be revealed by measured low CBF within certain brain regions through ASL. However, partial volume effects (PVE) mainly caused by signal cross-contamination due to pixel heterogeneity and limited spatial resolution of ASL, often prevents CBF from being precisely measured. Inaccurate CBF is prone to mislead and even deteriorate dementia disease diagnosis results, thereafter. In this paper, a novel dementia disease diagnosis strategy based on ASL is proposed for the first time. The diagnosis strategy is composed of two steps: 1) to conduct pixel-wise PVE correction on original ASL images and 2) to predict dementia disease severities based on corrected ASL images via ranking. Extensive experiments and comprehensive statistical analysis are carried out to demonstrate the superiority of the new strategy with comparison to several existing ones. Promising results are reported from the statistical point of view.

Initial Image Selection and its Influence on Super-Resolution Reconstruction

Super-resolution (SR) reconstruction is a technique to yield a higher resolution (HR) image from aliasing low resolution (LR) ones. An LR image is upsampled as the initialization, and then iteratively corrected in comparison with the other LR images. As the solution satisfying the SR constraints is non-unique, it is impossible to recover the original HR details completely by SR techniques. The solution reconstructed is sensitive to the starting point, especially when LR observations are insufficient, and may converge to a local optimum point. SR images reconstructed with different initializations may diverge in different ways from the true HR image. The influence of the initial HR estimate has not been sufficiently addressed so far by existing SR methods. We will explore this initial image selection issue to improve the performance of SR reconstruction.

A novel marker-less lung tumor localization strategy on low-rank fluoroscopic images with similarity learning

Fluoroscopic images depicting the movement of lung tumor lesions along with patients’ respirations are essential in contemporary image-guided lung cancer radiotherapy, as the accurate delivery of radiation dose on lung tumor lesions can be facilitated with the help of fluoroscopic images. However, the quality of fluoroscopic images is often not high, and several factors including image noise, artifact, ribs occlusion often prevent the tumor lesion from being accurate localized. In this study, a novel marker-less lung tumor localization strategy is proposed. Unlike conventional lung tumor localization strategies, it doesn’t require placing external surrogates on patients or implanting internal fiducial markers in patients. Thus ambiguous movement correlations between moving tumor lesions and surrogates as well as the risk of patients pneumothorax can be totally avoided. In this new strategy, fluoroscopic images are first decomposed into low-rank and sparse components via the split Bregman method, and then spectral clustering techniques are incorporated for similarity learning to realize the tumor localization task. Clinical data obtained from 60 patients with lung tumor lesions is utilized for experimental evaluation, and promising results obtained by the new strategy are demonstrated from the statistical point of view.