Gorkem Saygili

Quality assessment of asymmetric stereo video coding

It is well known that the human visual system can perceive high frequencies in 3D, even if that information is present in only one of the views. Therefore, the best 3D stereo quality may be achieved by asymmetric coding where the reference (right) and auxiliary (left) views are coded at unequal PSNR. However, the questions of what should be the level of this asymmetry and whether asymmetry should be achieved by spatial resolution reduction or SNR (quality) reduction are open issues. Extensive subjective tests indicate that when the reference view is encoded at sufficiently high quality, the auxiliary view can be encoded above a tow-quality threshold without a noticeable degradation on the perceived stereo video quality. This low-quality threshold may depend on the 3D display; e.g., it is about 31 dB for a parallax barrier display and 33 dB for a polarized projection display. Subjective tests show that, above this PSNR threshold value, users prefer SNR reduction over spatial resolution reduction on both parallax barrier and polarized projection displays. It is also observed that, if the auxiliary view is encoded below this threshold value, symmetric coding starts to perform better than asymmetric coding in terms of perceived 3D video quality.

3D display dependent quality evaluation and rate allocation using scalable video coding

It is well known that the human visual system can perceive high frequency content in 3D, even if that information is present in only one of the views. Then, the best 3D perception quality may be achieved by allocating the rates of the reference (right) and auxiliary (left) views asymmetrically. However the question of whether the rate reduction for the auxiliary view should be achieved by spatial resolution reduction (coding a downsampled version of the video followed by upsampling after decoding) or quality (QP) reduction is an open issue. This paper shows that which approach should be preferred depends on the 3D display technology used at the receiver. Subjective tests indicate that users prefer lower quality (larger QP) coding of the auxiliary view over lower resolution coding if a ¿full spatial resolution¿ 3D display technology (such as polarized projection) is employed. On the other hand, users prefer lower resolution coding of the auxiliary view over lower quality coding if a ¿reduced spatial resolution¿ 3D display technology (such as parallax barrier - autostereoscopic) is used. Therefore, we conclude that for 3D IPTV services, while receiving full quality/resolution reference view, users should subscribe to differently scaled versions of the auxiliary view depending on their 3D display technology. We also propose an objective 3D video quality measure that takes the 3D display technology into account.

Improving segment based stereo matching using SURF key points

State-of-the-art stereo matching algorithms estimate disparities using local block-matching, and subsequently refine the disparity estimates by introducing smoothness constraints and performing global energy minimization. Such algorithms are hampered by the inability of local block-matching algorithms to deal with repetitive patterns. This paper presents an approach that overcomes this problem by incorporating the disparity obtained from matching SURF key points between stereo image pairs. The algorithm provides further robustness to problems with repetitive pattern by penalizing the discrepancy between the initial and final disparity estimates in the global energy minimization. Evaluation of our approach on the Middleburry data set results shows that the our approach is more robust against repetitive patterns than existing approaches.

Adaptive stereo similarity fusion using confidence measures

We propose similarity fusion strategy based on stereo confidences.We propose a consensus strategy to exploit spatial correlation between pixels.Our fusion increases the accuracy of global and local stereo algorithms.We out-perform other fusion strategies. In most stereo-matching algorithms, stereo similarity measures are used to determine which image patches in a left-right image pair correspond to each other. Different similarity measures may behave very differently on different kinds of image structures, for instance, some may be more robust to noise whilst others are more susceptible to small texture variations. As a result, it may be beneficial to use different similarity measures in different image regions. We present an adaptive stereo similarity measure that achieves this via a weighted combination of measures, in which the weights depend on the local image structure. Specifically, the weights are defined as a function of a confidence measure on the stereo similarities: similarity measures with a higher confidence at a particular image location are given higher weight. We evaluate the performance of our adaptive stereo similarity measure in both local and global stereo algorithms on standard benchmarks such as the Middlebury and KITTI data sets. The results of our experiments demonstrate the potential merits of our adaptive stereo similarity measure.

Hybrid Kinect Depth Map Refinement for Transparent Objects

Depth sensors such as Kinect fail to find the depth of transparent objects which makes 3D reconstruction of such objects a challenge. The refinement algorithms for Kinect depth maps either do not address transparency or they only provide sparse depth on such objects which is inadequate for dense 3D reconstruction. In order to solve this problem, we propose a fully-connected CRF based hybrid refinement algorithm. We incorporate stereo cues from cross-modal stereo between IR and RGB cameras of the Kinect and Kinects depth map. Our algorithm does not require any additional cameras and still provides dense depth estimations of transparent objects and specular surfaces with high accuracy.

Accuracy Estimation for Medical Image Registration Using Regression Forests

This paper reports a new automatic algorithm to estimate the misregistration in a quantitative manner. A random regression forest is constructed, predicting the local registration error. The forest is built using local and modality independent features related to the registration precision, the transformation model and intensity-based similarity after registration. The forest is trained and tested using manually annotated corresponding points between pairs of chest CT scans. The results show that the mean absolute error of regression is 0.72 ± 0.96 mm and the accuracy of classification in three classes (correct, poor and wrong registration) is 93.4 %, comparing favorably to a competing method. In conclusion, a method was proposed that for the first time shows the feasibility of automatic registration assessment by means of regression, and promising results were obtained.

Accelerating Cost Aggregation for Real-Time Stereo Matching

Real-time stereo matching, which is important in many applications like self-driving cars and 3-D scene reconstruction, requires large computation capability and high memory bandwidth. The most time-consuming part of stereo-matching algorithms is the aggregation of information (i.e. costs) over local image regions. In this paper, we present a generic representation and suitable implementations for three commonly used cost aggregators on many-core processors. We perform typical optimizations on the kernels, which leads to significant performance improvement (up to two orders of magnitude). Finally, we present a performance model for the three aggregators to predict the aggregation speed for a given pair of input images on a given architecture. Experimental results validate our model with an acceptable error margin (an average of 10.4%). We conclude that GPU-like many-cores are excellent platforms for accelerating stereo matching.

Confidence Estimation for Medical Image Registration Based On Stereo Confidences

In this paper, we propose a novel method to estimate the confidence of a registration that does not require any ground truth, is independent from the registration algorithm and the resulting confidence is correlated with the amount of registration error. We first apply a local search to match patterns between the registered image pairs. Local search induces a cost space per voxel which we explore further to estimate the confidence of the registration similar to confidence estimation algorithms for stereo matching. We test our method on both synthetically generated registration errors and on real registrations with ground truth. The experimental results show that our confidence measure can estimate registration errors and it is correlated with local errors.

Stereo Similarity Metric Fusion Using Stereo Confidence

Stereo confidence measures are one of the most popular research topics in stereo vision. These measures give an indication about the certainty of the matching. The main aim of using confidence measures is to filter the erroneous disparity estimations at the end of the matching process. However, they can also be incorporated at the initial step of the matching process to obtain accurate estimations before the cost aggregation. In this paper, we propose to utilize stereo confidence measures for fusing different similarity measures in order to obtain robust estimations for aggregation. Since stereo similarity measures perform differently in varying conditions, the confidence-guided fusion of them makes stereo matching more robust against errors. We evaluate the performance of our algorithm in comparison to different similarity measures on the Middleburry benchmark stereo test set. The results show significant improvements on the accuracy of initial disparity estimations with our fusion strategy compared to different similarity measures.

Local-search based prediction of medical image registration error

Medical image registration is a crucial task in many different medical imaging applications. Hence, considerable amount of work has been published recently that aim to predict the error in a registration without any human effort. If provided, these error predictions can be used as a feedback to the registration algorithm to further improve its performance. Recent methods generally start with extracting image-based and deformation-based features, then apply feature pooling and finally train a Random Forest (RF) regressor to predict the real registration error. Image-based features can be calculated after applying a single registration but provide limited accuracy whereas deformation-based features such as variation of deformation vector field may require up to 20 registrations which is a considerably high time-consuming task. This paper proposes to use extracted features from a local search algorithm as image-based features to estimate the error of a registration. The proposed method comprises a local search algorithm to find corresponding voxels between registered image pairs and based on the amount of shifts and stereo confidence measures, it predicts the amount of registration error in millimetres densely using a RF regressor. Compared to other algorithms in the literature, the proposed algorithm does not require multiple registrations, can be efficiently implemented on a Graphical Processing Unit (GPU) and can still provide highly accurate error predictions in existence of large registration error. Experimental results with real registrations on a public dataset indicate a substantially high accuracy achieved by using features from the local search algorithm.