Anil Aksay

GrabcutD: improved grabcut using depth information

Popular state of the art segmentation methods such as Grab cut include a matting technique to calculate the alpha values for boundaries of segmented regions. Conventional Grabcut relies only on color information to achieve segmentation. Recently, there have been attempts to improve Grabcut using motion in video sequences. However, in stereo or multi-view analysis, there is additional information that could be also used to improve segmentation. Clearly, depth based approaches bear the potential discriminative power of ascertaining whether the object is nearer of farer. In this work, we propose and evaluate a Grabcut segmentation technique based on combination of color and depth information. We show the usefulness of the approach when stereo information is available and evaluate it using standard datasets against state of the art results.

A multi-modal dance corpus for research into interaction between humans in virtual environments

We present a new, freely available, multimodal corpus for research into, amongst other areas, real-time realistic interaction between humans in online virtual environments. The specific corpus scenario focuses on an online dance class application scenario where students, with avatars driven by whatever 3D capture technology is locally available to them, can learn choreographies with teacher guidance in an online virtual dance studio. As the dance corpus is focused on this scenario, it consists of student/teacher dance choreographies concurrently captured at two different sites using a variety of media modalities, including synchronised audio rigs, multiple cameras, wearable inertial measurement devices and depth sensors. In the corpus, each of the several dancers performs a number of fixed choreographies, which are graded according to a number of specific evaluation criteria. In addition, ground-truth dance choreography annotations are provided. Furthermore, for unsynchronised sensor modalities, the corpus also includes distinctive events for data stream synchronisation. The total duration of the recorded content is 1xa0h and 40xa0min for each single sensor, amounting to 55xa0h of recordings across all sensors. Although the dance corpus is tailored specifically for an online dance class application scenario, the data is free to download and use for any research and development purposes.

Motion and Disparity Aided Stereoscopic Full Frame Loss Concealment Method

Stereoscopic video is one of the emerging research areas especially among the video coding community. Along with the studies for efficiently compressing the stereoscopic and multiview video, new error concealment and error protection methods are also necessary to overcome the problems due to erroneous channel conditions in practical applications. In this paper we propose a full frame loss concealment algorithm for stereoscopic sequences. The proposed method uses redundancy and disparity between the two views and motion information between the previously decoded frames to estimate the lost frame. The results show that, the proposed algorithm outperforms the monoscopic methods when they are applied to the same view as they are simulcast coded.

Optimization of encoding and error protection parameters for 3D Video Broadcast over DVB-H

In this study, we propose a heuristic methodology for modeling the end-to-end distortion characteristics of an error resilient broadcast system for 3D video over Digital Video Broadcasting - Handheld (DVB-H).We also use this model to optimally select the parameters of the video encoder and the error correction scheme, namely, Multi Protocol Encapsulation - Forward Error Correction (MPE-FEC), minimizing the overall distortion. The proposed method models the RQ curve of video encoder and performance of channel codec to jointly derive the optimal encoder bit rates and unequal error protection (UEP) rates specific to the 3D video broadcast. Moreover, the distortion on the 3D video quality caused by packet losses and the loss rate of the channel is estimated. Finally, with the use of analytical models and estimated single packet loss distortions, end-to-end distortions are minimized and optimal encoder bit rates and UEP rates are obtained.

Evaluation of disparity map characteristics for stereo image coding

In order to compress stereo image pairs effectively, disparity compensation is the most widely used method. In this paper we examined the effects of using different disparity maps and their properties in an embedded JPEG2000 based disparity compensated stereo image coder. These properties include the block size, estimation method and the resulting entropy of the disparity map. Experimental results show that basic block matching gives better results than ground truth, especially on occluded regions and boundaries.

A new cost effective 3D measurement audit and model comparison system for verification tasks

A new unified system application for the production audit in an aerospace industry is presented in this paper which comprises two key application tools such as (a) 3D PAMT (production audit measurement tool) and (b) 3D PACT (production audit compare tool). In spite of the facts that above functionalities are modular wise independent, commonly they are related in terms of assisting the production audit task. 3D PAMT facilitates the verification of manufactured parts to be within a pre-defined threshold range using a calibrated stereo camera with the safety test engineer interaction in order to select the matching disparity points. The distance between datum points with or without reference to a planar reference surface model can be obtained. We describe the system flow, plus validate the technique via a number of experimental datasets. 3D PACT allows the identification of discrepancies between a computed 3D point cloud model and the corresponding digital mock-up point cloud model. Usually, the computer aided geometry model is built before an actual installation. This knowledge about the components of an installation assembly is available as semantic information in an extendable markup language (XML) format of the CATIA model. We have provided an use case study of a sample assembly with components such as cube, pyramid, rectangular prism and triangular prism. The proposed cost-effective and robust framework for 3D measurement audit and model comparison is based on the input available from a digital camera and the semantic metadata knowledge available from geometry models which can be used for verification tasks.

Stereoscopic video streaming with end-to-end modeling

In this work, we propose a stereoscopic video streaming system that uses rateless codes (Raptor codes) as the error protection scheme. Initially, we model the rate-distortion (RD) curve of video encoder and performance of channel codec. Then, we estimated the distortion on the stereoscopic video quality caused by packet losses. Finally, analytical models and estimated single packet loss distortions are used to minimize the end-to-end distortion and obtain optimal encoder bit rates and unequal error protection (UEP) rates. The simulation results clearly demonstrates the significant quality gain against the non-optimized schemes.

Importance of side information generation for distributed video coding

Distributed Video Coding (DVC) is a new coding paradigm, based on two information-theoretic results, which allows exploiting source statistics at the decoder only. This architecture, therefore, enables very simple encoders to be used in video coding. Wyner-Ziv video coding is a particular case of DVC which deals with lossy source coding where side information is available at the decoder only. In this paper, the performance improvement of Wyner-Ziv coding by using better side information generated by motion compensated frame interpolation algorithms proposed in literature is examined.