Cevahir Cigla

Efficient edge-preserving stereo matching

A computationally efficient stereo matching algorithm is introduced providing high precision dense disparity maps via local aggregation approach. The proposed algorithm exploits a novel paradigm, namely separable successive weighted summation (SWS) among horizontal and vertical directions with constant operational complexity, providing effective connected 2D support regions based on local color similarities. The intensity adaptive aggregation enables crisp disparity maps which preserve object boundaries and depth discontinuities. The same procedure is also utilized to diffuse information through overlapped pixels during occlusion handling. According to the experimental results on Middlebury online stereo benchmark, the proposed method is one of the most effective local stereo algorithm providing high quality disparity models by unifying constant time filtering and weighted aggregation. Hence, the proposed algorithm provides a competitive alternative, with its efficient GPU and FPGA implementations, for various local methods in terms of achieving precise disparity maps from stereo video within fast execution time.

Watermarking of Free-view Video

With the advances in image based rendering (IBR) in recent years, generation of a realistic arbitrary view of a scene from a number of original views has become cheaper and faster. One of the main applications of this progress has emerged as free-view TV(FTV), where TV-viewers select freely the viewing position and angle via IBR on the transmitted multiview video. Noting that the TV-viewer might record a personal video for this arbitrarily selected view and misuse this content, it is apparent that copyright and copy protection problems also exist and should be solved for FTV. In this paper, we focus on this newly emerged problem by proposing a watermarking method for free-view video. The watermark is embedded into every frame of multiple views by exploiting the spatial masking properties of the human visual system. Assuming that the position and rotation of the virtual camera is known, the proposed method extracts the watermark successfully from an arbitrarily generated virtual image. In order to extend the method for the case of an unknown virtual camera position and rotation, the transformations on the watermark pattern due to image based rendering operations are analyzed. Based upon this analysis, camera position and homography estimation methods are proposed for the virtual camera. The encouraging simulation results promise not only a novel method, but also a new direction for watermarking research.

Region-Based Dense Depth Extraction from Multi-View Video

A novel multi-view region-based dense depth map estimation problem is presented, based on a modified plane-sweeping strategy. In this approach, the whole scene is assumed to be region-wise planar. These planar regions are defined by back-projections of the over-segmented homogenous color regions on the images and the plane parameters are determined by angle-sweeping at different depth levels. The position and rotation of the plane patches are estimated robustly by minimizing a segment-based cost function, which considers occlusions, as well. The quality of depth map estimates is measured via reconstruction quality of the conjugate views, after warping segments into these views by the resulting homographies. Finally, a greedy-search algorithm is applied to refine the reconstruction quality and update the plane equations with visibility constraint. Based on the simulation results, it is observed that the proposed algorithm handles large un-textured regions, depth discontinuities at object boundaries, slanted surfaces, as well as occlusions.

Information permeability for stereo matching

A novel local stereo matching algorithm is introduced to address the fundamental challenge of stereo algorithms, accuracy and computational complexity dilemma. The time consuming intensity dependent aggregation procedure of local methods is improved in terms of both speed and precision. Providing connected 2D support regions, the proposed approach exploits a new paradigm, namely separable successive weighted summation (SWS) among horizontal and vertical directions enabling constant operational complexity. The weights are determined by four-neighborhood intensity similarity of pixels and utilized to model the information transfer rate, permeability, towards the corresponding direction. The same procedure is also utilized to diffuse information through overlapped pixels during occlusion handling after detecting unreliable disparity assignments. Successive weighted summation adaptively cumulates the support data based on local characteristics, enabling disparity maps to preserve object boundaries and depth discontinuities. According to the experimental results on Middlebury stereo benchmark, the proposed method is one of the most effective local stereo algorithm providing high quality disparity models by unifying constant time filtering and weighted aggregation. Hence, the proposed algorithm provides a competitive alternative for various local methods in terms of achieving precise and consistent disparity maps from stereo video within fast execution time.

Efficient graph-based image segmentation via speeded-up turbo pixels

An efficient graph based image segmentation algorithm exploiting a novel and fast turbo pixel extraction method is introduced. The images are modeled as weighted graphs whose nodes correspond to super pixels; and normalized cuts are utilized to obtain final segmentation. Utilizing super pixels provides an efficient and compact representation; the graph complexity decreases by hundreds in terms of node number. Connected K-means with convexity constraint is the key tool for the proposed super pixel extraction. Once the pixels are grouped into super pixels, iterative bi-partitioning of the weighted graph, as introduced in normalized cuts, is performed to obtain segmentation map. Supported by various experiments, the proposed two stage segmentation scheme can be considered to be one of the most efficient graph based segmentation algorithms providing high quality results.

Free-View Watermarking for Free-View Television

The recent advances in image based rendering (IBR) has pioneered a new technology, free-view television, in which TV-viewers select freely the viewing position and angle by the application of IBR on the transmitted multi-view video. Noting that the TV-viewer might also record a personal video for this arbitrarily selected view and misuse this content, it is apparent that copyright and copy protection problems also exist and should be solved for free-view TV. In this paper, we focus on this problem by proposing a watermarking method for free-view video. The watermark is embedded into every frame of multiple views by exploiting the spatial masking properties of the human visual system (HVS). Assuming that the position and rotation for the imagery view is known, the proposed method extracts the watermark successfully from an arbitrarily generated image. In order to extend the method for the case of an unknown imagery camera position and rotation, the modifications on the watermark pattern due to image based rendering operations are also analyzed. Based on this analysis, a camera position and homography estimation method is proposed considering the operations in image based rendering. The results show that the watermark detection is achieved successfully for the cases in which the imagery camera is arbitrarily located on the camera plane.

Region-based image segmentation via graph cuts

A graph theoretic color image segmentation algorithm is proposed, in which the popular normalized cuts image segmentation method is improved with modifications on its graph structure. The image is represented by a weighted undirected graph, whose nodes correspond to over-segmented regions, instead of pixels, that decreases the complexity of the overall algorithm. In addition, the link weights between the nodes are calculated through the intensity similarities of the neighboring regions. The irregular distribution of the nodes, as a result of such a modification, causes a bias towards combining regions with high number of links. This bias is removed by limiting the number of links for each node. Finally, segmentation is achieved by bipartitioning the graph recursively according to the minimization of the normalized cut measure. The simulation results indicate that the proposed segmentation scheme performs quite faster than the traditional normalized cut methods, as well as yielding better segmentation results due to its region-based representation.

Segmentation in multi-view video via color, depth and motion cues

In the light of dense depth map estimation, motion estimation and object segmentation, the research on multi-view video (MW) content has becoming increasingly popular due to its wide application areas in the near future. In this work, object segmentation problem is studied by additional cues due to depth and motion fields. Segmentation is achieved by modeling images as graphical models and performing popular normalized cuts method with some modifications. In the graphical models, each node is represented by a group of pixels, instead of individual pixels, which are obtained as a result of over-segmentation of the images. These over-segmented regions are also utilized in the dense depth map estimation step; in which 3D planar models are assigned for each of these sub-regions. Moreover, optical flow is estimated based on afline motion assumption for these regions. The links of the graphical models are weighted according to the depth, motion and color similarities of the pixel groups due to these regions. Once the links are obtained, segmentation is achieved by recursively bi-partitioning the graph via removing the weak links. Experiments indicate that the proposed framework achieves precise segmentation results for MVV sequences.

Temporally consistent dense depth map estimation via Belief Propagation

A method for estimating temporally and spatially consistent dense depth maps in multiple camera setups is presented which is important for reduction of perception artifacts in 3D displays. For this purpose, initially, depth estimation is performed for each camera with the piece-wise planarity assumption and Markov Random Field (MRF) based relaxation at each time instant independently. During the relaxation step, the consistency of depth maps for different cameras is also considered for the reliability of the models. Next, temporal consistency of the depth maps is achieved in two steps. In the first step, median filtering is applied for the static or background pixels, whose intensity levels are constant in time. Such an approach decreases the number of inconsistent depth values significantly. The second step considers the moving pixels and MRF formulation is updated by the additional information from the depth maps of the consequent frames through motion compensation. For the solution of the MRF formulation for both spatial and temporal consistency, Belief Propagation approach is utilized. The experiments indicate that the proposed method provide reliable dense depth map estimates both in spatial and temporal domains.

Depth Assisted Object Segmentation in Multi-View Video

In this work, a novel and unified approach for multi-view video (MVV) object segmentation is presented. In the first stage, a region-based graph-theoretic color segmentation algorithm is proposed, in which the popular normalized cuts segmentation method is improved with some modifications on its graph structure. Segmentation is obtained by recursive bi-partitioning of a weighted graph of an initial over-segmentation mask. The available segmentation mask is also utilized during dense depth map estimation step, based on a novel modified plane- and angle- sweeping strategy for each of these regions. Dense depth estimation is achieved by region-wise planarity assumption for the whole scene, in which depth models are estimated for sub-regions. Finally, the multi-view image segmentation algorithm is extended to object segmentation in MVV by the additional optical flow information. The required motion field is obtained via region- based matching that has consistent parameterization with color segmentation and dense depth map estimation algorithms. Experimental results indicate that proposed approach segments semantically meaningful objects in MVV with high precision.