Yong Ju Jung

Visual comfort assessment metric based on salient object motion information in stereoscopic video

Objective assessment of visual comfort for stereoscopic video is of great importance for stereoscopic image safety issue. We propose a novel visual comfort assessment metric framework that systematically exploits human visual attention models. In a stereoscopic video shot, perceptually significant regions where human subjects pay more attention are likely to play an essential role in determining the overall level of visual comfort. As a specific example of this concept, we develop a visual comfort metric that quantifies the level of visual discomfort caused by fast salient object motion. The performance of the proposed visual comfort metric has been evaluated using natural stereoscopic videos. The experimental results show that the proposed visual comfort metric significantly improves the correlations with subjective judgment.

Predicting Visual Discomfort of Stereoscopic Images Using Human Attention Model

We introduce a new objective assessment method for visual discomfort of stereoscopic images that makes effective use of the human visual attention model. The proposed method takes into account visual importance regions that play an important role in determining the overall degree of visual discomfort of a stereoscopic image. After obtaining a saliency-based visual importance map for an image, perceptually significant disparity features are extracted to predict the overall degree of visual discomfort. Experimental results show that the proposed method can achieve significantly higher prediction accuracy than the state-of-the-art methods.

Predicting Visual Discomfort Using Object Size and Disparity Information in Stereoscopic Images

This paper proposes object-dependent disparity features to predict the visual discomfort in stereoscopic 3-D images. The proposed object-dependent disparity features quantify the level of visual comfort influenced by disparity gradient of nearby objects and object width, respectively. They consist of relative disparity (mean of disparity difference between nearby objects) and object thickness (ratio of mean width to mean absolute disparity of an object). The prediction performance of the proposed disparity features is evaluated using various types of stereoscopic images. Experimental results demonstrate that the combined use of the proposed object-dependent disparity features substantially improve the prediction performance of the conventional disparity magnitude- and spatial complexity-related features. The performance gain ranges from 0.045 to 0.135 of correlation coefficient, compared with the feature combinations used in the conventional visual comfort metrics.

Visual Importance- and Discomfort Region-Selective Low-Pass Filtering for Reducing Visual Discomfort in Stereoscopic Displays

In human vision, the depth of focus (DOF) blur is important to perceive single binocular vision. Unfortunately, conventional stereoscopic displays cannot provide a natural DOF blur effect. Synthetic DOF blur that simulates the natural DOF blur effect could be useful for reducing the visual discomfort in stereoscopic displays. However, the incorrect introduction of artificial blur in the image might also have negative effects. This paper proposes a selective DOF blur technique that is applied only to local regions that induce high visual discomfort but are less visually important. We employ a visual importance analysis to obtain reasonable estimates of the areas/objects that are likely to be attended using saliency maps. After identifying the regions likely to cause discomfort because of excessive relative disparity, we apply low-pass filtering for the less-salient neighboring pixels of the discomfort regions. We demonstrate that the DOF blur simulation for less important background regions that induce visual discomfort is helpful in reducing the visual discomfort in stereoscopic viewing.

Attention model-based visual comfort assessment for stereoscopic depth perception

This paper investigates visual discomfort induced by depth perception for viewing stereoscopic 3D images. In order to assess the degree of visual discomfort, we exploit an attention model-based visual comfort metric, considering perceptually significant regions of stereoscopic images. In particular, the relationship between visual comfort and depth perception is modeled based on psychophysical experiments with diverse amount of binocular disparities. Then, several spatial pooling techniques are applied using a visual comfort model in order to investigate the feasibility of the attention-model based visual comfort metric. Subjective assessments have been conducted using real stereoscopic 3D images, demonstrating the efficiency of the attention model-based visual comfort metric.

Human brain response to visual fatigue caused by stereoscopic depth perception

In stereopsis, excessive screen disparity is known as one of the main causes of visual fatigue. Although the visual fatigue caused by stereoscopic depth perception has been reported in a large number of subjective assessments, there has been little effort to measure objective indicators for the visual fatigue. In this study, we investigated the relationship between the visual fatigue and the amount of binocular disparity by using functional magnetic resonance imaging (fMRI). As a result, we detected strong neuronal activities in frontal eye field (FEF) when the presented visual stimuli had disparities beyond the comfort zone. Since FEF is known to be involved in eye movement control, it also appears to have correlation with the visual fatigue caused by the stereoscopic depth perception.

Visual Comfort Amelioration Technique for Stereoscopic Images: Disparity Remapping to Mitigate Global and Local Discomfort Causes

This paper proposes a new disparity remapping framework to improve the visual comfort of stereoscopic images. The proposed framework adaptively remaps disparities of a scene according to different causes of visual discomfort. A linear disparity remapping is first performed in order to address visual discomfort induced by excessive disparities. This linear remapping changes the disparities of the scene to obtain an overall target disparity range. Then, a nonlinear disparity remapping process selectively adjusts the disparity of problematic local disparity ranges according to their contribution to the visual discomfort. The proposed nonlinear disparity remapping process enables us to minimize the loss in perceived depth range while further improving visual comfort. The effectiveness of the proposed disparity remapping framework has been successfully evaluated by subjective assessments of visual comfort and naturalness. Experimental results demonstrate the validity of the proposed remapping framework. More importantly, we show that the nonlinear refinement of disparity in problematic regions can efficiently improve visual comfort while maintaining the naturalness of the scene.

Visual comfort improvement in stereoscopic 3D displays using perceptually plausible assessment metric of visual comfort

Visual discomfort is caused by several factors such as accommodation-vergence conflict and window violation in conventional stereoscopic displays. To ensure comfortable conditions in the stereoscopic viewing, it is essential to develop an effective way of improving visual comfort in stereoscopic displays. This paper proposes a new method for visual comfort improvement that adjusts the disparity of a scene under the guidance of an objective assessment metric of visual comfort. The proposed method employs a visual comfort assessment metric that takes into account various discomfort factors in the stereoscopic viewing. The results of subjective assessment experiments demonstrate that the proposed method can efficiently improve visual comfort in the stereoscopic viewing.

Visual discomfort induced by fast salient object motion in stereoscopic video

This paper investigates visual discomfort induced by fast motion of salient object in a stereoscopic video. We have conducted a subjective assessment to investigate the degree of visual discomfort caused by motion characteristics of a controlled graphics object in a video scene. As results of the subjective assessment, we observe the changes of the degree of visual discomfort with varying velocity and direction of object motion. In order to verify the acceptability of our observation for real stereoscopic 3D videos, we exploit the concept of visual saliency to define the salient object motion severely affecting the degree of visual discomfort in a video scene. The salient object motion feature is extracted and a visual comfort model is derived from our observation. Then we predict the degree of visual discomfort by using the extracted motion feature and the visual comfort model. We have conducted a subjective test to compare the predicted visual comfort score with actual subjective score. The experiment results show that the predicted visual comfort score correlates well with the actual subject score.

Inter-view consistent hole filling in view extrapolation for multi-view image generation

This paper proposes a new inter-view consistent hole filling method in view extrapolation for multi-view image generation. In stereopsis, inter-view consistency regarding structure, color, and luminance is one of the crucial factors that affect the overall viewing quality of three-dimensional image contents. In particular, the inter-view inconsistency could induce visual stress on the human visual system. To ensure the inter-view consistency, the proposed method suggests a hole filling method in an order from the nearest to farthest view to the reference view by propagating the filled color information in the preceding view. In addition, a novel depth map filling method is incorporated to achieve the inter-view consistency. Experimental results show that the proposed method significantly improves the inter-view consistency for multiview images and depth maps, compared to those of previous methods.