Jun Zhou

Quaternion wavelet phase based stereo matching for uncalibrated images

This paper proposes a quaternion wavelet phase based stereo matching (QWPSM) scheme for uncalibrated image pairs. In this scheme, we estimate the disparity by directly establishing correspondences between quaternionic phase structures of two quaternion wavelet filtered (QWF) images. Firstly, linear-phase quaternion wavelet filters (LPQWFs) are constructed from real biorthogonal wavelet bases. Then, quaternion phases are extracted under each scale through quaternion wavelet filtering of the multiscale transformed image pyramids. The disparity estimation is formed as a minimization process of a local energy weighted cost function, and propagated from coarse to fine scales. Costs can adaptively alleviate the negative effects of phase singularities, which are the main causes of mismatches in phase-based stereo matching. Multiscale matching strategy is used to avoid phase wrapping and improve convergence speed. Experimental results are promising in various image pairs.

No-Reference Quality Assessment of Screen Content Pictures

Recent years have witnessed a growing number of image and video centric applications on mobile, vehicular, and cloud platforms, involving a wide variety of digital screen content images. Unlike natural scene images captured with modern high fidelity cameras, screen content images are typically composed of fewer colors, simpler shapes, and a larger frequency of thin lines. In this paper, we develop a novel blind/no-reference (NR) model for accessing the perceptual quality of screen content pictures with big data learning. The new model extracts four types of features descriptive of the picture complexity, of screen content statistics, of global brightness quality, and of the sharpness of details. Comparative experiments verify the efficacy of the new model as compared with existing relevant blind picture quality assessment algorithms applied on screen content image databases. A regression module is trained on a considerable number of training samples labeled with objective visual quality predictions delivered by a high-performance full-reference method designed for screen content image quality assessment (IQA). This results in an opinion-unaware NR blind screen content IQA algorithm. Our proposed model delivers computational efficiency and promising performance. The source code of the new model will be available at: https://sites.google.com/site/guke198701/publications.

Binocular mismatch induced by luminance discrepancies on stereoscopic images

Luminance discrepancies between image pairs occur owing to inconsistent parameters between stereoscopic camera devices and from imperfect capture conditions. Such discrepancies induce binocular mismatches and affect the visual comfort that is felt by viewers, as well as their ability to fuse stereoscopic. To better understand and observe this effect, we built a stereoscopic images database of 240 luminance discrepancy images and 30 natural images with subjective scores of visual discomfort and fusion difficulty. Two features, binocular contrast and luminance similarity were extracted to analyze the relationship between the subjective scores and the luminance discrepancies. Structural dissimilarity and average luminance are used to predict the effects of binocular mismatches. The experimental results show that the combination of binocular contrast, structural dissimilarity and average luminance exhibits high consistency with subjective scores of visual discomfort, fusion difficulty and overall binocular mismatches in terms of Spearmans Rank Ordered Correlation Coefficient.

2D phase-based matching in uncalibrated images

A novel 2D phase-based matching approach is proposed to resolve the general stereo image matching problem. It is different from the current uncalibrated matching techniques, most of which obtain 2D dense disparity map after the epipolar geometry has been recovered. In this paper, the disparity is directly estimated by simply establishing correspondences between quaternionic phase structures of two QWF (quaternion wavelet filtered) images. Real and short-length biorthogonal wavelet bases are exploited to build linear-phase quaternion wavelet filters (LPQWFs). Once phases are extracted from the QWF image pair, the disparity estimation is formed as a minimization process of a cost function, which is formulated as a similarity measure for comparing quaternion wavelet phases. With regard to the mismatches near phase singularities, phase stability constraints are imposed on cost aggregation stage. And multi-scale matching strategy is introduced to avoid phase wrapping problem and improve convergence speed. The experimental results are encouraging in various image pairs.

Adaptive high-frequency clipping for improved image quality assessment

It is widely known that the human visual system (HVS) applies multi-resolution analysis to the scenes we see. In fact, many of the best image quality metrics, e.g. MS-SSIM and IW-PSNR/SSIM are based on multi-scale models. However, in existing multi-scale type of image quality assessment (IQA) methods, the resolution levels are fixed. In this paper, we examine the problem of selecting optimal levels in the multi-resolution analysis to preprocess the image for perceptual quality assessment. According to the contrast sensitivity function (CSF) of the HVS, the sampling of visual information by the human eyes approximates a low-pass process. For images, the amount of information we can extract depends on the size of the image (or the object(s) inside) as well as the viewing distance. Therefore, we proposed a wavelet transform based adaptive high-frequency clipping (AHC) model to approximate the effective visual information that enters the HVS. After the high-frequency clipping, rather than processing separately on each level, we transform the filtered images back to their original resolutions for quality assessment. Extensive experimental results show that on various databases (LIVE, IVC, and Toyama-MICT), performance of existing image quality algorithms (PSNR and SSIM) can be substantially improved by applying the metrics to those AHC model processed images.

Visual discomfort prediction on stereoscopic 3D images without explicit disparities

Almost all existing 3D visual discomfort prediction models are based, at least in part, on features that are extracted from computed disparity maps. These include such estimated quantities such as the maximum disparity, disparity range, disparity energy and other measures of the disparity distribution. A common first step when implementing a 3D visual discomfort model is some form of disparity calculation, whence the accuracy of prediction largely depends on the accuracy of the disparity result. Unfortunately, most algorithms that compute disparity maps are expensive, and are not guaranteed to deliver sufficiently accurate or perceptually relevant disparity data. This raises the question of whether it is possible to build a 3D discomfort prediction model without explicit disparity calculation. Towards this possibility, we have developed a new feature map, called the percentage of un-linked pixels (PUP), that is descriptive of the presence of disparity, and which can be used to accurately predict experienced 3D visual discomfort without the need for actually calculating disparity values. Instead, PUP features are extracted by predicting the percentage of un-linked pixels in corresponding retinal patches of image pairs. The un-linked pixels are determined by feature classification on orientation and luminance distributions. Calculation of PUP maps is much faster than traditional disparity computation, and the experimental results demonstrate that the predictive power attained using the PUP map is highly competitive with prior models that rely on computed disparity maps. HighlightsA first of a kind 3D discomfort model without disparity calculation is proposed.A new feature map, the percentage of un-linked pixels is developed in the model.PUP map is superior to prior models that rely on computed disparity maps.

Survey on Algorithm and VLSI Architecture for MPEG-Like Video Coder

Efficient and dedicated hardware architecture and accelerator micro-engines are crucial implementation forms of MPEG-like video coder. It is significant to excavate and generalize the common technologies and design philosophy of hardwired MPEG-like coders behind number of architectures from academic and industrial communities. This paper makes systematic survey on algorithm and architecture of hardwired MPEG-like coders, from microscopic and macroscopic perspectives, taking H.264/AVC as the analysis target. Recent advances in hardware architectures of prevailing H.264/AVC coders are reviewed and summarized. Furthermore, important algorithm modules, such as integer and fractional pixel motion estimation, mode decision, motion vector prediction, intra prediction, rate control, CABAC coder and deblocking filter are reviewed with detailed analysis on algorithm and hardware architecture. In accordance with the intrinsic characteristics of the algorithm flows, the major design constraints and consideration factors of algorithm and architecture are analyzed respectively. The common technologies of the prevailing architectures are summarized from a systematic perspective, coving different levels ranging from algorithm, architecture, to control and data flows, etc. Based on these analysis, this survey further highlights in-depth summarization and perspectives on MPEG-like coder architecture design. First, the design challenges with multiple target performance optimization are analyzed, and the possible solutions for design challenges are systematically summarized. Second, the rate-distortion-complexity constrained algorithm optimization for MPEG-like video encoder is discussed. Third, typical four-level hierarchical architecture model (SoC system, module, inter-connection, memory) is analyzed, and the pivotal memory architecture and inter-connection architecture are emphasized for analysis. Moreover, the algorithm and architecture design suggestions and preferences for the vital modules are discussed. Fourth, the composite performances of prevailing architectures are evaluated. The concerned target parameters including hardware logic cost, SRAM size, external memory bandwidth, throughput efficiency, power dissipation, and rate-distortion performance are taken as comparison factors. Finally, this paper provides explicit perspectives on future trends of video coder architecture design. The proposed paper can be taken as design reference for H.264/AVC coder hardware architecture, and offer further insight into algorithm and architecture optimization for the new emerging HEVC standard.

On evaluation the quality of subjective S3D comfort assessment

Stereoscopic 3D (S3D) image technology has been extensively developed in the last decades. Visual discomfort such as eye strain, headache, fatigue, asthenopia, and other phenomena leading to a less pleasant viewing experience is still a potential issue in S3D applications. How to evaluate S3D image quality that related to visual discomfort is still a challenging problem. A larger number of studies have been done on S3D Image Quality Assessment (S3D IQA) where the subjective assessed S3D Image Databases play an important role. The subjective scores were collected for each S3D image in database with a number of viewers. Usually, Likert scale is adopted for observers to mark their subjective quality score, and then mean opinion score (MOS) is estimated. Due to the law of comparative judgment, the quality of subjective scores varies among observers and depends on the judgment method. This paper studied the quality of two subjective assessment methodologies — single stimulus (SS) and pairwise comparison (PC). Considering the S3D IQA as a S3D images quality ranking problem, we applied single stimulus and pairwise comparison subjective testing on a set of S3D images with known geometric distortions. From SS subjective testing results, the S3D images ranking can be derived by sorting MOSs directly. From PC subjective testing results, the ranking can be derived from DMOS scores. The distorted S3D images can be ranked via their geometric distortion parameters. The quality of subjective assessed results from SS and PC are then evaluated on the correlation between their ranking results to corresponding geometric distortions. With the collected MOSs for geometric distorted S3D image database, a deep-learning based S3D IQA model was used to study the relationship between the model performance and the quality of subjective assessment.

APOLIPOROTEIN E POLYMORPHISM IN CHINESE POPULATION WITH VARIOUS TYPES OF COGNITIVE DISORDERS

Background:Having an APOE-e4 allele is a risk factor for Alzheimer’s Disease (AD), but some data also suggest that it may have protective effects earlier in life. Low education is another known risk factor for AD. We examined whether parental education interacted with risk genes for AD to predict later cognitive performance. Methods: Participants were 1048 white, non-Hispanic community-dwelling men of European ancestry average age 62. Genetic risk was evaluated with two indicators: the APOE genotype and an AD polygenic risk score (AD-PRS) derived from the International Genomics of Alzheimer’s Project data. Here we used the AD-PRS excluding SNPs in the APOE region. APOEe4 status was categorized as having no e4 allele (e4-; 71%) versus any (e4+; 29%). Parental education was operationalized as having at least one (76%) or neither parent (24%) complete high school. We controlled for non-independence of twins in mixed models, adjusted for the first 3 principal components from the SNP data, and age. Cognitive performance was assessed with a measure of general cognitive ability at ages 20 and 62 and nine specific cognitive abilities at age 62. Results:Both parental education and the parental education by APOE genotype interaction were significantly associated with GCA at ages 20 and 62, as well as with five out of nine cognitive abilities at age 62 (Abstract Reasoning, Episodic Memory, Processing Speed, Working Memory, and Visual Spatial Ability). The interactions indicated that being e4+ when neither parent had a high school education was associated with significantly lower cognitive ability scores; however, when one parent had at least a high school education, participants who were ε4+ showed better cognitive function than their ε4counterparts, even including general cognitive ability at age 20. The AD-PRS by parental education interaction was significant for only three cognitive measures. Conclusions: As expected, presence of the APOE-e4 allele under disadvantaged childhood conditions was associated with poorer performance. Consistent with the differential susceptibility hypothesis, however, in more favorable contexts the presence of an ε4 allele appeared to be advantageous. In addition, the age 20 results suggest that these cognitive differences were apparent relatively early in development.

Stereoscopic images quality assessment based on deep learning

With the popularity of stereoscopic 3D (S3D) images and videos, many advanced objective quality assessment methods have been proposed to evaluate viewers Quality of Experience (QoE). Among them, most algorithms take advantages of the disparity maps to extract useful features. On the other hand, deep learning has been one of the hottest research topics during these years, but limited efforts focused on the field in objective quality evaluation of S3D images. In this paper, we propose a S3D image quality assessment (S3D IQA) method based on deep learning. In this method, the Convolutional Restricted Boltzmann Machines (CRBM) combined with Factored Third-Order RBM (FTO-RBM) is considered as learning model to extract feature maps from pre-processed left and right images automatically. Then an improved traversal algorithm based on two pooling strategies is put forward to optimize extracted feature maps, which improves the final quality assessment performance significantly. Experimental results show that our S3D IQA method achieves good performance on 3D databases tested.