Zuzana Cernekova

Information theory-based shot cut/fade detection and video summarization

New methods for detecting shot boundaries in video sequences and for extracting key frames using metrics based on information theory are proposed. The method for shot boundary detection relies on the mutual information (MI) and the joint entropy (JE) between the frames. It can detect cuts, fade-ins and fade-outs. The detection technique was tested on the TRECVID2003 video test set having different types of shots and containing significant object and camera motion inside the shots. It is demonstrated that the method detects both fades and abrupt cuts with high accuracy. The information theory measure provides us with better results because it exploits the inter-frame information in a more compact way than frame subtraction. It was also successfully compared to other methods published in literature. The method for key frame extraction uses MI as well. We show that it captures satisfactorily the visual content of the shot.

Shot detection in video sequences using entropy based metrics

A new method for detecting shot boundaries in video sequences using metrics based on information theory is proposed. The method relies on the mutual information and the joint entropy between frames and can detect cuts, fade-ins and fade-outs. The detection technique was tested on TV video sequences having different types of shots and significant object and camera motion inside the shots. It was favorably compared to other recently proposed shot cut detection techniques. The method is proven to detect both fades and abrupt cuts very effectively.

Video shot segmentation using singular value decomposition

A new method for detecting shot boundaries in video sequences using singular value decomposition (SVD) is proposed. The method relies on performing singular value decomposition on the matrix A created from 3D histograms of single frames. We have used SVD for its capabilities to derive a low dimensional refined feature space from a high dimensional raw feature space, where pattern similarity can easily be detected. The method can detect cuts and gradual transitions, such as dissolves and fades, which cannot be detected easily by entropy measures.

Entropy metrics used for video summarization

New methods for detecting shot boundaries in video sequences and for extracting key frames using metrics based on information theory are proposed. The method for shot cut detection relies on the mutual information and the joint entropy between the frames. It can detect cuts, fade-ins and fade-outs. The detection technique was tested on TV video sequences having different types of shots and containing significant object and camera motion inside the shots. It is demonstrated that the method detects both fades and abrupt cuts with high accuracy. The method for key frame extraction is using the mutual information. We show that it captures satisfactorily the visual content of the shot.

Video shot-boundary detection using singular-value decomposition and statistical tests

We deal with video shot-cut detection in digital videos using the singular-value decomposition (SVD). SVD is performed on a matrix whose columns are the 3D frame color histograms. We have used SVD for its capabilities to derive a refined low-dimensional feature space from the high-dimensional raw feature space, where similar video patterns are placed together and can be easily clustered. After SVD is performed, a two-phase process is employed to detect the shots. In the first phase, a dynamic clustering method is used to create the frame clusters. In the second phase, every two consecutive clusters, obtained by the clustering procedure, are tested for a possible merging in order to reduce false shot-cut detections. In the merging phase, statistical hypothesis testing is used. The detection technique was applied to several TRECVID video test sets that exhibit different types of shots and contain significant object and camera motion inside the shots. We demonstrate that the method detects cuts and gradual transitions, such as dissolves and fades, with high accuracy.

Temporal Video Segmentation by Graph Partitioning

A novel temporal video segmentation method that, in addition to abrupt cuts, can detect with very high accuracy gradual transitions such as dissolves, fades and wipes is proposed. The method relies on evaluating mutual information between multiple pairs of frames within a certain temporal frame window. This way we create a graph where the frames are nodes and the measures of similarity correspond to the weights of the edges. By finding and disconnecting the weak connections between nodes we separate the graph to subgraphs ideally corresponding to the shots. Experiments on TRECVID2004 video test set containing different types of shot transitions and significant object and camera motion inside the shots prove that the method is very efficient

Video shot segmentation using fusion of SVD and mutual information features

A new method for detecting shot boundaries in video sequences by fusing features obtained by singular value decomposition (SVD) and mutual information (MI) is proposed. The first method relies on performing singular value decomposition on a matrix created from 3D color histograms of single frames. The method can detect cuts and gradual transitions, such as dissolves, fades and wipes. The second method relies on evaluating mutual information between two consecutive frames. It can detect abrupt cuts, fade-ins and fade-outs with very high accuracy. A combination of features derived from these methods and subsequent processing through a clustering procedure results in very efficient detection of abrupt cuts and gradual transitions, as demonstrated by experiments on the TRECVID2004 video test set containing different types of shots with significant object and camera motion inside the shots.

Single camera pointing gesture recognition for interaction in edutainment applications

In this paper, a method for recognizing pointing gestures without markers is proposed. The video-based system uses one camera only, which observes the user in front of a large screen and identifies the 2D position pointed by him/her on this screen, his/her arm being in the fully extended position towards the screen. A GVF-snake is used in order to detect the pointing hand of the user, which is tracked in the following frames using the particle filters tracker. The center of gravity of the snake is used as a feature point and is transformed using linear transformation directly into the canvas coordinates. The method was tested on a large screen using applications designed for a wide range of different and even technically unversed users such as an image exploration for a virtual museum exhibit or intuitive interaction applications for gaming purposes. Experiments show very promising results for recognizing the pointing gestures by using a single camera.

Best view methods suitability for different types of objects

In this paper, we present three approaches to the best view selection for 3D objects, in order to find the proper combination. The first method is based on geometry and uses the maximum amount of visible vertices. The second method uses the visual attention model and the third one calculates the amount of information in a scene after projecting the 3D scene to the 2D image using entropy. Several 3D models from Blender are used in the experiment. We compare these different approaches and explore their suitability on various objects. An online questionnaire is made to find out which of the tested images fits better to the human tastes.

Study of the best view selection based on a human observer

In this paper we present results from a questionnaire focused on searching for the best view on virtual 3D objects. For our research we collected a database of virtual models from different categories and we tried to find out if the best view selected by respondents varies from category to category. We performed a clustering using the von Mises-Fisher mixture model to find out the means of each group of selected views. Further we would like to determine whether we can predict the best view similar to the one chosen by the questionnaire just by categorizing examined object.