Zenonas Theodosiou

MuLVAT: A Video Annotation Tool Based on XML-Dictionaries and Shot Clustering

Recent advances in digital video technology have resulted in an explosion of digital video data which are available through the Web or in private repositories. Efficient searching in these repositories created the need of semantic labeling of video data at various levels of granularity, i.e., movie, scene, shot, keyframe, video object, etc. Through multilevel labeling video content is appropriately indexed, allowing access from various modalities and for a variety of applications. However, despite the huge efforts for automatic video annotation human intervention is the only way for reliable semantic video annotation. Manual video annotation is an extremely laborious process and efficient tools developed for this purpose can make, in many cases, the true difference. In this paper we present a video annotation tool, which uses structured knowledge, in the form of XML dictionaries, combined with a hierarchical classification scheme to attach semantic labels to video segments at various level of granularity. Video segmentation is supported through the use of an efficient shot detection algorithm; while shots are combined into scenes through clustering with the aid of a Genetic Algorithm scheme. Finally, XML dictionary creation and editing tools are available during annotation allowing the user to always use the semantic label she/he wishes instead of the automatically created ones.

POLYBIO: Multimodal Biometric Data Acquisition Platform and Security System

Biometrics is the automated method of recognizing a person based on a physiological or behavioural characteristic. Biometric technologies are becoming the foundation of an extensive array of highly secure identification and personal verification solutions. In the last few years there is increasing evidence that technologies based on multimodal biometrics can provide better identification results if proper fusion schemes are accommodated. In this work, we present a novel platform for multimodal biometric acquisition which combines voice, video, fingerprint and palm photo acquisition through an integrated device, and the preliminary fusion experiments on combining the acquired biometrics modalities. The results are encouraging and show clear improvement both in terms of False Acceptance Rate and False Rejection Rates compared to the corresponding single modality approaches. In the current report, fusion was accommodated at the output of the single modalities; however, fusion experimentation is ongoing and further fusion methodologies are under investigation.

Web Image Context Extraction Based on Semantic Representation of Web Page Visual Segments

Among the most challenging scientific interests of the past years, special attention has been given to the task of web image information mining. Web images exist in huge amounts on the web and several methods for their efficient description and representation have been proposed so far. In many of the exploited algorithms, web image information is extracted from textual sources such as image file names, anchor texts, existing keywords and, of course, surrounding text. However, the systems that attempt to mine information for images using surrounding text suffer from several problems, such as the inability to correctly assign all relevant text to an image and discard the irrelevant text at the same time. A novel method for indexing web images is discussed in the present paper. The proposed system uses visual cues in order to obtain a web page segmentation. The segments are represented with semantic metrics and a k-means clustering assigns these segments to the web image they refer to. The evaluation procedure indicates that the semantic representation method of the visual segments delivers a good description for the web images.

Crowdsourcing annotation: Modelling keywords using low level features

Tagging large collections is often prohibitive and manual tags are known to be imprecise, ambiguous, inconsistent and subject to many variations. A possible way to alleviate these problems and improve the annotation quality is to obtain multiple annotations per image by assigning several annotators into the task. In the current work we present an approach to model the view of several annotators using four MPEG-7 descriptors and a well known data classifier. We apply keywords modelling to the annotation data collected in the framework of Commandaria project where sixteen non-expert users annotated a set of a hundred images using a predefined set of keywords. The images sharing a common keyword are grouped together and used for the creation of the visual model corresponds to this keyword. Finally, the created models used to classify the images into the keyword classes in terms of 2-classes combinations using the 10-fold cross-validation technique. The experimental results are examined under two perspectives: First, in terms of the separation ability of the various keyword classes and second, in terms of the efficiency of the four visual descriptors as far as the image classification task is concerned.

Evaluating Annotators Consistency with the Aid of an Innovative Database Schema

Automatic semantic tagging of multimedia is still inefficient due to the difficulties in modelling abstract or complex terms using low level features. The degree of consensus and homogeneity in judgements among annotators is very important in semantic image and video retrieval. In this paper we present a novel method in evaluating the annotators consistency, which uses an innovative database schema and combines two different annotation approaches. A set of 100 images were annotated by 16 annotators using vocabulary keywords and free keywords. The results indicate that combination of annotation methods may lead to increased annotation consistency compared to a single method but this is not a general fact. As expected the use of free keywords and images require tagging that is not directly related to their content, lead to increase the annotators inconsistency.

Spatial histogram of keypoints (SHIK)

Among a variety of feature extraction approaches, special attention has been given to the SIFT algorithm which delivers good results for many applications. However, the non fixed and huge dimensionality of the extracted SIFT feature vector cause certain limitations when it is used in machine learning frameworks. In this paper, we introduce Spatial Histogram of Keypoints (SHiK), which keeps the spatial information of localized keypoints, on an effort to overcome this limitation. The proposed technique partitions the image into a fixed number of ordered sub-regions based on the Hilbert space filling curve and counts the localized keypoints found inside each sub-region. The resulting spatial histogram is a compact and discriminative low-level feature vector that shows significantly improved performance on classification tasks. The proposed method achieves high accuracy on different datasets and performs significantly better on scene datasets compared to the Spatial Pyramid Matching method.

Modelling Crowdsourcing Originated Keywords within the Athletics Domain

Image classification arises as an important phase in the overall process of automatic image annotation and image retrieval. Usually, a set of manually annotated images is used to train supervised systems and classify images into classes. The act of crowdsourcing has largely focused on investigating strategies for reducing the time, cost and effort required for the creation of the annotated data. In this paper we experiment with the efficiency of various classifiers in building visual models for keywords through crowdsourcing with the aid of Weka tool and a variety of low-level features. A total number of 500 manually annotated images related to athletics domain are used to build and test 8 visual models. The experimental results have been examined using the classification accuracy and are very promising showing the ability of the visual models to classify the images into the corresponding classes with the highest average classification accuracy of 74.38% in the purpose of SMO data classifier.

Semantic Gap between People: An Experimental Investigation Based on Image Annotation

Image annotation still remains the method of preference in multimedia search despite the development of many content-based multimedia retrieval platforms. Manual annotation is an extremely labour-intensive and time consuming task while the annotation expresses the view of a particular annotator at a specific context and time. Although the semantic gap has attracted large amount of research interest, the age and gender gaps in manual annotation have not been examined in detail. The aim of this study was to explore the gender and age differences in (1) the way of annotating images and, (2) the inter-annotator agreement. Our questionnaire based survey was conducted using 40 Cypriot citizens divided into two age groups who were asked to annotate an image dataset using a vocabulary of 52 keywords. Our results indicate that there are age differences in the way people annotate images, while the gender differences are smaller than our assumptions. Furthermore, there is an adequate agreement among participants for both age and gender groups.

Early malicious activity discovery in microblogs by social bridges detection

With the emerging and intense use of Online Social Networks (OSNs) amongst young children and teenagers (youngsters), safe networking and socializing on the Web has faced extensive scrutiny. Content and interactions which are considered safe for adult OSN users might embed potentially threatening and malicious information when it comes to underage users. This work is motivated by the strong need to safeguard youngsters OSNs experience such that they can be empowered and aware. The topology of a graph is studied towards detecting the so called “social bridges”, i.e. the major supporters of malicious users, who have links and ties to both honest and malicious user communities. A graph-topology based classification scheme is proposed to detect such bridge linkages which are suspicious for threatening youngsters networking. The proposed scheme is validated by a Twitter network, at which potentially dangerous users are identified based on their Twitter connections. The achieved performance is higher compared to previous efforts, despite the increased complexity due to the variety of groups identified as malicious.

Airborne asbestos fibers detection in microscope images using re-initialization free active contours

Breathing in asbestos fibers can lead to a number of diseases, the fibers become trapped in the lung and cannot be removed by either coughing or the persons immune system. Atmospheric concentrations of carcinogenic asbestos fibers, have traditionally been measured visually using phase contrast microscopy. However, because this measurement method requires great skill, and has poor reproducibility and objectivity, the development of automatic counting methods has been long anticipated. In this paper we proposed an automated fibers detection method based on a variational formulation of geometric active contours that forces the level set function to be close to signed distance function and therefore completely eliminates the need of the costly re-initialization procedure. The method was evaluated using a ground truth of 29 manually annotated images. The results were encouraging for the further development of the proposed method.