Muhammad Shahid Farid

Depth image based rendering with inverse mapping

Three-dimensional video has gained much attention during the last decade due its vast applications in cinema, television, animation and virtual reality. The design of intermediate view synthesis algorithms that are efficient both in terms of computational complexity and visual quality is a paramount goal in the fields of 3D free view point television and displays. This papers focuses on the design of a low complexity view synthesis algorithm that produces better quality of the virtual image. A novel view synthesis technique to create a virtual view from two video sequences with corresponding depths is proposed. The technique employs low complexity integer pixel precision warping and a novel approach for hole filling based on inverse mapping. The proposed technique is tested over a number of video sequences and compared with existing state of the art methods, yielding excellent results both in terms of signal to noise ratio and visual quality.

Panorama View With Spatiotemporal Occlusion Compensation for 3D Video Coding

The future of novel 3D display technologies largely depends on the design of efficient techniques for 3D video representation and coding. Recently, multiple view plus depth video formats have attracted many research efforts since they enable intermediate view estimation and permit to efficiently represent and compress 3D video sequences. In this paper, we present spatiotemporal occlusion compensation with panorama view (STOP), a novel 3D video coding technique based on the creation of a panorama view and occlusion coding in terms of spatiotemporal offsets. The panorama picture represents the most of the visual information acquired from multiple views using a single virtual view, characterized by a larger field of view. Encoding the panorama video with state-of-the-art HECV and representing occlusions with simple spatiotemporal ancillary information STOP achieves high-compression ratio and good visual quality with competitive results with respect to competing techniques. Moreover, STOP enables free viewpoint 3D TV applications whilst allowing legacy display to get a bidimensional service using a standard video codec and simple cropping operations.

Multiple human detection in depth images

Most human detection algorithms in depth images perform well in detecting and tracking the movements of a single human object. However, their performance is rather poor when the person is occluded by other objects or when there are multiple humans present in the scene. In this paper, we propose a novel human detection technique which analyzes the edges in depth image to detect multiple people. The proposed technique detects a human head through a fast template matching algorithm and verifies it through a 3D model fitting technique. The entire human body is extracted from the image by using a simple segmentation scheme comprising a few morphological operators. Our experimental results on three large human detection datasets and the comparison with the state-of-the-art method showed an excellent performance achieving a detection rate of 94.53% with a small false alarm of 0.82%.

Image de-fencing framework with hybrid inpainting algorithm

Detection and removal of fences from digital images become essential when an important part of the scene turns to be occluded by such unwanted structures. Image de-fencing is challenging because manually marking fence boundaries is tedious and time-consuming. In this paper, a novel image de-fencing algorithm that effectively detects and removes fences with minimal user input is presented. The user is only requested to mark few fence pixels; then, color models are estimated and used to train Bayes classifier to segment the fence and the background. Finally, the fence mask is refined exploiting connected component analysis and morphological operators. To restore the occluded region, a hybrid inpainting algorithm is proposed that integrates exemplar-based technique with a pyramid-based interpolation approach. In contrast to previous solutions which work only for regular pattern fences, the proposed technique is able to remove both regular and irregular fences. A large number of experiments are carried out on a wide variety of images containing different types of fences demonstrating the effectiveness of the proposed approach. The proposed approach is also compared with state-of-the-art image de-fencing and inpainting techniques and showed convincing results.

Objective quality metric for 3D virtual views

In free-viewpoint television (FTV) framework, due to hardware and bandwidth constraints, only a limited number of viewpoints are generally captured, coded and transmitted; therefore, a large number of views needs to be synthesized at the receiver to grant a really immersive 3D experience. It is thus evident that the estimation of the quality of the synthesized views is of paramount importance. Moreover, quality assessment of the synthesized view is very challenging since the corresponding original views are generally not available either on the encoder (not captured) or the decoder side (not transmitted). To tackle the mentioned issues, this paper presents an algorithm to estimate the quality of the synthesized images in the absence of the corresponding reference images. The algorithm is based upon the cyclopean eye theory. The statistical characteristics of an estimated cyclopean image are compared with the synthesized image to measure its quality. The prediction accuracy and reliability of the proposed technique are tested on standard video dataset compressed with HEVC showing excellent correlation results with respect to state-of-the-art full reference image and video quality metrics.

Edge enhancement of depth based rendered images

Depth image based rendering is a well-known technology for the generation of virtual views in between a limited set of views acquired by a cameras array. Intermediate views are rendered by warping image pixels based on their depth. Nonetheless, depth maps are usually imperfect as they need to be estimated through stereo matching algorithms; moreover, for representation and transmission requirements depth values are obviously quantized. Such depth representation errors translate into a warping error when generating intermediate views thus impacting on the rendered image quality. We observe that depth errors turn to be very critical when they affect the object contours since in such a case they cause significant structural distortion in the warped objects. This paper presents an algorithm to improve the visual quality of the synthesized views by enforcing the shape of the edges in presence of erroneous depth estimates. We show that it is possible to significantly improve the visual quality of the interpolated view by enforcing prior knowledge on the admissible deformations of edges under projective transformation. Both visual and objective results show that the proposed approach is very effective.

A panoramic 3D video coding with directional depth aided inpainting

The success of 3D and free-viewpoint television largely depends on the efficient representation and compression of 3D video in addition to viable rendering methods. This paper presents a novel 3D video coding technique based on the creation of a panorama view to compact the information of a stereoscopic pair. The panorama view represents the information that would be visible to a virtual camera with a larger field of view embracing all the available views. The information in the panorama view is then used to estimate any intermediate view using depth image based rendering. Furthermore, to fill the disocclusions in the reconstructed view a directional depth aided fast marching inpainting technique is presented. The panorama view and corresponding depth map are amenable to standard video compression. In this paper we show that using the novel HEVC standard the proposed 3D video format can be compressed very efficiently.

Edges shape enforcement for visual enhancement of depth image based rendering

Depth image based rendering of intermediate views with high visual quality remains a challenging goal in presence of estimated and quantized depth values. Among the other rendering artifacts we observed that edges are usually affected by significant warping errors. In particular, because of depth estimation inaccuracy around object boundaries the edges may completely loose their original shape during the warping process. Nonetheless, edges represent one of the most important cues for the human visual system. In this paper a novel technique aiming at improving the edge rendering is presented. As opposed to previous approaches, the technique exploits only texture information, thus avoiding possible errors in depth estimation. The idea is based on the enforcement of prior knowledge of the edge shape under projective transformation. The proposed algorithm works in two steps: first the damaged edges of the warped image are detected, then these latter are corrected so as to better approximate their shape in the reference view. Finally the corrected edges are rendered within the intermediate image without introducing noticeable texture artifacts. The proposed algorithm has been tested on a variety of standard video sequences exhibiting excellent results in terms of rendered image visual quality.

Content Based Image Retrieval Using Localized Multi-texton Histogram

This paper presents a simple yet efficient image retrieval technique that defines image feature descriptors using localized multi-texton histogram. The proposed technique extracts a unique feature vector for each image in the image database based on its shape, texture and color. First, the image is divided into smaller equal size blocks and then for each block texture orientation is computed independently. Second, each block is filtered with a set of predefined textons and finally, a co-occurrence relation is established from the orientation and the filtered text on image. This relationship in turn provides a powerful feature vector. To retrieve similar images, the feature vector of the query image is computed and compared with the feature vectors of the stored images using Euclidean distance measure. The proposed algorithm is tested on standard image dataset Corel 1000 for accuracy and recall with favorable results. It is also compared with existing state of the art Context Based Image Retrieval algorithm and showed convincing results.

DOST: a distributed object segmentation tool

This paper presents a novel distributed object segmentation framework that allows one to extract potentially large coherent objects from digital images. The proposed approach requires minimum user supervision and permits to segment the objects accurately. It works in three steps starting with the user input in form of few mouse clicks on the target object. First, based on user input, the statistical characteristics of the target distributed object are modeled with Gaussian mixture model. This model serves as the primary segmentation of the object. In the second step, the segmentation result is refined by performing connected component analysis to reduce false positives. In the final step the resulting segmentation map is dilated to select the neighboring pixels that are potentially incorrectly classified; this allows us to recast the segmentation as a graph partitioning problem that can be solved using the well-known graph cut technique. Extensive experiments have been carried out on heterogeneous images to test the accuracy of the proposed method for the segmentation of various types of distributed objects. Examples of application of proposed technique in remote sensing to segment roads and rivers from aerial images are also presented. The visual and objective evaluation and comparison with the existing techniques show that the proposed tool can deliver optimal performance when applied to tough object segmentation tasks.