Chenwei Deng

Just Noticeable Difference for Images With Decomposition Model for Separating Edge and Textured Regions

In just noticeable difference (JND) models, evaluation of contrast masking (CM) is a crucial step. More specifically, CM due to edge masking (EM) and texture masking (TM) needs to be distinguished due to the entropy masking property of the human visual system. However, TM is not estimated accurately in the existing JND models since they fail to distinguish TM from EM. In this letter, we propose an enhanced pixel domain JND model with a new algorithm for CM estimation. In our model, total-variation based image decomposition is used to decompose an image into structural image (i.e., cartoon like, piecewise smooth regions with sharp edges) and textural image for estimation of EM and TM, respectively. Compared with the existing models, the proposed one shows its advantages brought by the better EM and TM estimation. It has been also applied to noise shaping and visual distortion gauge, and favorable results are demonstrated by experiments on different images.

Image Retargeting Quality Assessment: A Study of Subjective Scores and Objective Metrics

This paper presents the result of a recent large-scale subjective study of image retargeting quality on a collection of images generated by several representative image retargeting methods. Owning to many approaches to image retargeting that have been developed, there is a need for a diverse independent public database of the retargeted images and the corresponding subjective scores to be freely available. We build an image retargeting quality database, in which 171 retargeted images (obtained from 57 natural source images of different contents) were created by several representative image retargeting methods. And the perceptual quality of each image is subjectively rated by at least 30 viewers, meanwhile the mean opinion scores (MOS) were obtained. It is revealed that the subject viewers have arrived at a reasonable agreement on the perceptual quality of the retargeted image. Therefore, the MOS values obtained can be regarded as the ground truth for evaluating the quality metric performances. The database is made publicly available (Image Retargeting Subjective Database, [Online]. Available: http://ivp.ee.cuhk.edu.hk/projects/demo/retargeting/index.html) to the research community in order to further research on the perceptual quality assessment of the retargeted images. Moreover, the built image retargeting database is analyzed from the perspectives of the retargeting scale, the retargeting method, and the source image content. We discuss how to retarget the images according to the scale requirement and the source image attribute information. Furthermore, several publicly available quality metrics for the retargeted images are evaluated on the built database. How to develop an effective quality metric for retargeted images is discussed through a specifically designed subjective testing process. It is demonstrated that the metric performance can be further improved, by fusing the descriptors of shape distortion and content information loss.

Robust Image Coding Based Upon Compressive Sensing

Multiple description coding (MDC) is one of the widely used mechanisms to combat packet-loss in non-feedback systems. However, the number of descriptions in the existing MDC schemes is very small (typically 2). With the number of descriptions increasing, the coding complexity increases drastically and many decoders would be required. In this paper, the compressive sensing (CS) principles are studied and an alternative coding paradigm with a number of descriptions is proposed based upon CS for high packet loss transmission. Two-dimentional discrete wavelet transform (DWT) is applied for sparse representation. Unlike the typical wavelet coders (e.g., JPEG 2000), DWT coefficients here are not directly encoded, but re-sampled towards equal importance of information instead. At the decoder side, by fully exploiting the intra-scale and inter-scale correlation of multiscale DWT, two different CS recovery algorithms are developed for the low-frequency subband and high-frequency subbands, respectively. The recovery quality only depends on the number of received CS measurements (not on which of the measurements that are received). Experimental results show that the proposed CS-based codec is much more robust against lossy channels, while achieving higher rate-distortion (R-D) performance compared with conventional wavelet-based MDC methods and relevant existing CS-based coding schemes.

Extreme learning machines: new trends and applications

Extreme learning machine (ELM), as a new learning framework, draws increasing attractions in the areas of large-scale computing, high-speed signal processing, artificial intelligence, and so on. ELM aims to break the barriers between the conventional artificial learning techniques and biological learning mechanism and represents a suite of machine learning techniques in which hidden neurons need not to be tuned. ELM theories and algorithms argue that “random hidden neurons” capture the essence of some brain learning mechanisms as well as the intuitive sense that the efficiency of brain learning need not rely on computing power of neurons. Thus, compared with traditional neural networks and support vector machine, ELM offers significant advantages such as fast learning speed, ease of implementation, and minimal human intervention. Due to its remarkable generalization performance and implementation efficiency, ELM has been applied in various applications. In this paper, we first provide an overview of newly derived ELM theories and approaches. On the other hand, with the ongoing development of multilayer feature representation, some new trends on ELM-based hierarchical learning are discussed. Moreover, we also present several interesting ELM applications to showcase the practical advances on this subject.摘要创新点极限学习机作为一种全新的机器学习理论和框架, 在大数据计算、 高速信号处理, 人工智能等领域越来越受到关注。 极限学习机旨在打破传统学习理论和生物学习机制之间的壁垒, 该理论认为人脑的学习效率不依赖于单个神经元的计算能力, 因此极限学习机通过随机产生的隐层神经元来逼近大脑学习机理, 取得了比传统神经网络和支持向量机更高的学习精度、 更快的训练速度以及更少的人为干预。 本文对近年来提出的极限学习机新理论和新方法进行综述, 在此基础上重点介绍基于极限学习机的多层特征表征方面的最新研究成果, 最后介绍极限学习机的实际应用。

Robust image compression based on compressive sensing

The existing image compression methods (e.g., JPEG2000, etc.) are vulnerable to bit-loss, and this is usually tackled by channel coding that follows. However, source coding and channel coding have conflicting requirement. In this paper, we address the problem with an alternative paradigm, and a novel compressive sensing (CS) based compression scheme is therefore proposed. Discrete wavelet transform (DWT) is applied for sparse representation, and based on the property of 2-D DWT, a fast CS measurements taking method is presented. Unlike the unequally important discrete wavelet coefficients, the resultant CS measurements carry nearly the same amount of information and have minimal effects for bit-loss. At the decoder side, one can simply reconstruct the image via l1 minimization. Experimental results show that the proposed CS-based image codec without resorting to error protection is more robust compared with existing CS technique and relevant joint source channel coding (JSCC) schemes.

NMF-Based Image Quality Assessment Using Extreme Learning Machine

Numerous state-of-the-art perceptual image quality assessment (IQA) algorithms share a common two-stage process: distortion description followed by distortion effects pooling. As for the first stage, the distortion descriptors or measurements are expected to be effective representatives of human visual variations, while the second stage should well express the relationship among quality descriptors and the perceptual visual quality. However, most of the existing quality descriptors (e.g., luminance, contrast, and gradient) do not seem to be consistent with human perception, and the effects pooling is often done in ad-hoc ways. In this paper, we propose a novel full-reference IQA metric. It applies non-negative matrix factorization (NMF) to measure image degradations by making use of the parts-based representation of NMF. On the other hand, a new machine learning technique [extreme learning machine (ELM)] is employed to address the limitations of the existing pooling techniques. Compared with neural networks and support vector regression, ELM can achieve higher learning accuracy with faster learning speed. Extensive experimental results demonstrate that the proposed metric has better performance and lower computational complexity in comparison with the relevant state-of-the-art approaches.

Content-Based Image Compression for Arbitrary-Resolution Display Devices

The existing image coding methods cannot support content-based spatial scalability with high compression. In mobile multimedia communications, image retargeting is generally required at the user end. However, content-based image retargeting (e.g., seam carving) is with high computational complexity and is not suitable for mobile devices with limited computing power. The work presented in this paper addresses the increasing demand of visual signal delivery to terminals with arbitrary resolutions, without heavy computational burden to the receiving end. In this paper, the principle of seam carving is incorporated into a wavelet codec (i.e., SPIHT ). For each input image, block-based seam energy map is generated in the pixel domain. In the meantime, multilevel discrete wavelet transform (DWT) is performed. Different from the conventional wavelet-based coding schemes, DWT coefficients here are grouped and encoded according to the resultant seam energy map. The bitstream is then transmitted in energy descending order. At the decoder side, the end user has the ultimate choice for the spatial scalability without the need to examine the visual content; an image with arbitrary aspect ratio can be reconstructed in a content-aware manner based upon the side information of the seam energy map. Experimental results show that, for the end users, the received images with an arbitrary resolution preserve important content while achieving high coding efficiency for transmission.

Optimal Compression Plane for Efficient Video Coding

All existing video coding standards developed so far deem video as a sequence of natural frames (formed in the XY plane), and treat spatial redundancy (redundancy along X and Y directions) and temporal redundancy (redundancy along T direction) differently and separately. In this paper, we investigate into a new compression (redundancy reduction) method for video in which the frames are allowed to be formed in a non-XY plane. We are to exploit fuller extent of video redundancy, and propose an adaptive optimal compression plane determination process to be used as a preprocessing step prior to any standard video coding scheme. The essence of the scheme is to form the frames in the plane formed by two axes (among X, Y, and T) corresponding to signal correlation evaluation, which enables better prediction (therefore better compression). In spite of the simplicity of the proposed method, it can be used for both lossless and lossy compression, and with and without interframe prediction. Extensive experimental results show that the new coding method improves the performance of the video coding for a number of coding methods (inclusive of lossless and near-lossless Motion JPEG-LS, Motion JPEG, Motion JPG2K, H.264 intraonly profile, and H.264) and videos with different visual content.

Study of subjective and objective quality assessment of retargeted images

This paper presents the result of a recent large-scale subjective study of image retargeting quality on a collection of images generated by several representative image retargeting methods. Owning to many approaches to image retargeting that are developed, there is a need for a diverse independent public database of the retargeted images and the corresponding subjective scores that is freely available. We build an image retargeting quality database, in which 171 retargeted images (obtained from 57 natural source images of different contents) were generated by several representative image retargeting methods. The perceptual quality of each image is evaluated by at least 30 human subjects and the mean opinion scores (MOS) were recorded. Furthermore, several publicly available quality metrics for the retargeted images are evaluated on the built database. The database is made available [1] to the research community in order to further research on the perceptual quality assessment of the retargeted images.

Gradient-based no-reference image blur assessment using extreme learning machine

The increasing number of demanding consumer digital multimedia applications has boosted interest in no-reference (NR) image quality assessment (IQA). In this paper, we propose a perceptual NR blur evaluation method using a new machine learning technique, i.e., extreme learning machine (ELM). The proposed metric, Blind Image Blur quality Evaluator (BIBE), exploits scene statistics of gradient magnitudes to model the properties of blurred images, and then the underlying blur features are derived by fitting gradient magnitudes distribution. The resultant feature is finally mapped into an associated quality score using ELM. As subjective evaluation scores by human beings are integrated into training, machine learning techniques can predict image quality more accurately than those traditional methods. Compared with other learning techniques such as support vector machine (SVM), ELM has better learning performance and faster learning speed. Experimental results on public databases show that the proposed BIBE correlates well with human perceived blurriness, and outperforms the state-of-the-art specific NR blur evaluation metrics as well as generic NR IQA methods. Moreover, the application of automatic focusing system for digital cameras further confirms the capability of BIBE.