Klimis S. Ntalianis

Efficient summarization of stereoscopic video sequences

An efficient technique for summarization of stereoscopic video sequences is presented, which extracts a small but meaningful set of video frames using a content-based sampling algorithm. The proposed video-content representation provides the capability of browsing digital stereoscopic video sequences and performing more efficient content-based queries and indexing. Each stereoscopic video sequence is first partitioned into shots by applying a shot-cut detection algorithm so that frames (or stereo pairs) of similar visual characteristics are gathered together. Each shot is then analyzed using stereo-imaging techniques, and the disparity field, occluded areas, and depth map are estimated. A multiresolution implementation of the recursive shortest spanning tree (RSST) algorithm is applied for color and depth segmentation, while fusion of color and depth segments is employed for reliable video object extraction. In particular, color segments are projected onto depth segments so that video objects on the same depth plane are retained, while at the same time accurate object boundaries are extracted. Feature vectors are then constructed using multidimensional fuzzy classification of segment features including size, location, color, and depth. Shot selection is accomplished by clustering similar shots based on the generalized Lloyd-Max algorithm, while for a given shot, key frames are extracted using an optimization method for locating frames of minimally correlated feature vectors. For efficient implementation of the latter method, a genetic algorithm is used. Experimental results are presented, which indicate the reliable performance of the proposed scheme on real-life stereoscopic video sequences.

An efficient fully unsupervised video object segmentation scheme using an adaptive neural-network classifier architecture

In this paper, an unsupervised video object (VO) segmentation and tracking algorithm is proposed based on an adaptable neural-network architecture. The proposed scheme comprises: 1) a VO tracking module and 2) an initial VO estimation module. Object tracking is handled as a classification problem and implemented through an adaptive network classifier, which provides better results compared to conventional motion-based tracking algorithms. Network adaptation is accomplished through an efficient and cost effective weight updating algorithm, providing a minimum degradation of the previous network knowledge and taking into account the current content conditions. A retraining set is constructed and used for this purpose based on initial VO estimation results. Two different scenarios are investigated. The first concerns extraction of human entities in video conferencing applications, while the second exploits depth information to identify generic VOs in stereoscopic video sequences. Human face/ body detection based on Gaussian distributions is accomplished in the first scenario, while segmentation fusion is obtained using color and depth information in the second scenario. A decision mechanism is also incorporated to detect time instances for weight updating. Experimental results and comparisons indicate the good performance of the proposed scheme even in sequences with complicated content (object bending, occlusion).

EFFICIENT UNSUPERVISED CONTENT-BASED SEGMENTATION IN STEREOSCOPIC VIDEO SEQUENCES

This paper presents an efficient technique for unsupervised content-based segmentation in stereoscopic video sequences by appropriately combined different content descriptors in a hierarchical framework. Three main modules are involved in the proposed scheme; extraction of reliable depth information, image partition into color and depth regions and a constrained fusion algorithm of color segments using information derived from the depth map. In the first module, each stereo pair is analyzed and the disparity field and depth map are estimated. Occlusion detection and compensation are also applied for improving the depth map estimation. In the following phase, color and depth regions are created using a novel complexity-reducing multiresolution implementation of the Recursive Shortest Spanning Tree algorithm (M-RSST). While depth segments provide a coarse representation of the image content, color regions describe very accurately object boundaries. For this reason, in the final phase, a new segmentation fusion algorithm is employed which projects color segments onto depth segments. Experimental results are presented which exhibit the efficiency of the proposed scheme as content-based descriptor, even in case of images with complicated visual content.

Unsupervised semantic object segmentation of stereoscopic video sequences

In this paper, we present an efficient technique for unsupervised semantically meaningful object segmentation of stereoscopic video sequences. Using this technique we extract semantic objects using the additional information a stereoscopic pair of frames provides. Each pair is analyzed and the disparity field, occluded areas and depth map are estimated. The key algorithm, which is applied on the stereo pair of images and performs the segmentation, is a powerful low-complexity multiresolution implementation of the RSST algorithm. Color segment fusion is employed using the depth segments as a kind of constraint. Finally experimental results are presented which demonstrate the high-quality of semantic object segmentation this technique achieves.

Human face watermarking based on Zernike moments

A novel human face watermarking scheme is proposed in this paper, providing copyright protection of semantic content. To achieve this goal, skin detection is initially performed using a skin filter, which relies on color information and then, face extraction is achieved using a combination of a morphological filter and a human face template. An invariant watermark is then designed and tested against attacks using invariant Zernike moments. The proposed algorithm has the advantages of being robust, computationally efficient and overheads transmitted to the decoder side are very low. The performance of the proposed human face watermarking system is tested under various signal distortions such as JPEG lossy compression, blurring, filtering and cropping. Experimental results on real life images indicate the efficiency and robustness of the proposed scheme

Human action annotation, modeling and analysis based on implicit user interaction

This paper proposes an integrated framework for analyzing human actions in video streams. Despite most current approaches that are just based on automatic spatiotemporal analysis of sequences, the proposed method introduces the implicit user-in-the-loop concept for dynamically mining semantics and annotating video streams. This work sets a new and ambitious goal: to recognize, model and properly use “average user’s” selections, preferences and perception, for dynamically extracting content semantics. The proposed approach is expected to add significant value to hundreds of billions of non-annotated or inadequately annotated video streams existing in the Web, file servers, databases etc. Furthermore expert annotators can gain important knowledge relevant to user preferences, selections, styles of searching and perception.

An optimal interpolation-based scheme for video summarization

In this paper, an optimal and efficient algorithm for video summarization is proposed by exploiting temporal variations of video visual content. In particular, the most characteristic frames/shots (key-frames/shots) are extracted by estimating appropriate points on the feature vector curve, which represent in an optimal way the corresponding trajectory. This is performed by minimizing the approximation error of the feature vector curve and the respective curve formed by the estimated points using an interpolation scheme. A genetic algorithm is used for the minimization task, since the complexity of an exhaustive search is too large to be implemented. Furthermore, a fast technique for increasing the number of extracted key-frames/shots is presented.

An Optimized Key-Frames Extraction Scheme Based on SVD and Correlation Minimization

In this paper an optimized and efficient technique for keyframes extraction of video sequences is proposed, which leads to selection of a meaningful set of video frames for each given shot. Initially for each frame, the singular value decomposition method is applied and a diagonal matrix is produced, containing the singular values of the frame. Afterwards, a feature vector is created for each frame, by gathering the respective singular values. Next, all feature vectors of the shot are collected to form the feature vectors basin of this shot. Finally, a genetic algorithm approach is proposed and applied to the vectors basin, for locating frames of minimally correlated feature vectors, which are selected as keyframes. Experimental results indicate the promising performance of the proposed scheme on real life video shots

Multiresolution gradient vector flow field: a fast implementation towards video object plane segmentation

In this paper, an efficient scheme for video object segmentation is proposed. The scheme is based on a multiresolution Gradient Vector Flow field (M-GVF) and a Motion Geometric Space (MGS) formulation. In particular the proposed scheme is initialized from an object approximation which can be provided either (a) automatically (unsupervised case) based on a depth map estimation method or (b) semi-automatically by user interaction. In the following, several feature points are estimated on the initial object contour (i.e. depth object) and an M-GVF adapted MGS is created to determine the direction that a feature point is allowed to move to. In this framework, each feature point moves onto its MGS in order to locate the contour of the physical video object. Experimental results are presented to indicate the reliable performance of the proposed scheme on real life stereoscopic and monocular video sequences.

An active contour-based video object segmentation scheme for stereoscopic video sequences

A modified snake-based scheme is presented for unsupervised stereoscopic semantic segmentation. The scheme utilizes the provided depth information and the power of active contours to adjust to object edges. Each stereo pair is analyzed and a depth map is constructed. Then a multiresolution implementation of the recursive shortest spanning tree (RSST) segmentation algorithm is applied to the depth field to generate depth segments. Afterwards a novel edge map, free of several non-object edges, is constructed. The next step includes the initialization of the modified snake. The constructed edge map empowers the snake to move towards the object while, at the same time, its new bending energy decreases the computational complexity. Finally the active contour extracts the video object planes (VOP). Experimental results indicate the reliable performance of the proposed scheme on real life stereoscopic video sequences.