Mohan S. Kankanhalli

Multimodal fusion for multimedia analysis: a survey

This survey aims at providing multimedia researchers with a state-of-the-art overview of fusion strategies, which are used for combining multiple modalities in order to accomplish various multimedia analysis tasks. The existing literature on multimodal fusion research is presented through several classifications based on the fusion methodology and the level of fusion (feature, decision, and hybrid). The fusion methods are described from the perspective of the basic concept, advantages, weaknesses, and their usage in various analysis tasks as reported in the literature. Moreover, several distinctive issues that influence a multimodal fusion process such as, the use of correlation and independence, confidence level, contextual information, synchronization between different modalities, and the optimal modality selection are also highlighted. Finally, we present the open issues for further research in the area of multimodal fusion.

Shape measures for content based image retrieval: a comparison

Abstract A great deal of work has been done on the evaluation of information retrieval systems for alphanumeric data. The same thing can not be said about the newly emerging multimedia and image database systems. One of the central concerns in these systems is the automatic characterization of image content and retrieval of images based on similarity of image content. In this paper, we discuss effectiveness of several shape measures for content based similarity retrieval of images. The different shape measures we have implemented include outline based features (chain code based string features, Fourier descriptors, UNL Fourier features), region based features (invariant moments, Zernike moments, pseudo-Zernike moments), and combined features (invariant moments & Fourier descriptors, invariant moments & UNL Fourier features). Given an image, all these shape feature measures (vectors) are computed automatically, and the feature vector can either be used for the retrieval purpose or can be stored in the database for future queries. We have tested all of the above shape features for image retrieval on a database of 500 trademark images. The average retrieval efficiency values computed over a set of fifteen representative queries for all the methods is presented. The output of a sample shape similarity query using all the features is also shown.

Anonymous secure routing in mobile ad-hoc networks

Although there are a large number of papers on secure routing in mobile ad-hoc networks, only a few consider the anonymity issue. We define more strict requirements on the anonymity and security properties of the routing protocol, and notice that previous research works only provide weak location privacy and route anonymity, and are vulnerable to specific attacks. Therefore, we propose the anonymous secure routing (ASR) protocol that can provide additional properties on anonymity, i.e. identity anonymity and strong location privacy, and at the same time ensure the security of discovered routes against various passive and active attacks. Detailed analysis shows that ASR can achieve both anonymity and security properties, as defined in the requirements, of the routing protocol in mobile ad-hoc networks.

Color matching for image retrieval

Abstract Color is an important attribute for image matching and retrieval. We present two new color matching methods, the “Reference Color Table Method” and a “Distance Method”, for image retrieval. Both these methods and an existing method “Histogram Intersection” were implemented and tested for a database size of 170 color images. To compare the efficacy of each method, a figure of merit, called “Efficiency of Retrieval”, is defined. The results show that both the new methods perform better than the existing method, and that the Reference Color Table Method gives the best results.

Creating audio keywords for event detection in soccer video

This paper presents a novel framework called audio keywords to assist event detection in soccer video. Audio keyword is a middle-level representation that can bridge the gap between low-level features and high-level semantics. Audio keywords are created from low-level audio features by using support vector machine learning. The created audio keywords can be used to detect semantic events in soccer video by applying a heuristic mapping. Experiments of audio keywords creation and event detection based on audio keywords have illustrated promising results. According to the experimental results, we believe that audio keyword is an effective representation that is able to achieve more intuitionistic result for event detection in sports video compared with the method of event detection directly based on low-level features.

An eye fixation database for saliency detection in images

To learn the preferential visual attention given by humans to specific image content, we present NUSEF- an eye fixation database compiled from a pool of 758 images and 75 subjects. Eye fixations are an excellent modality to learn semantics-driven human understanding of images, which is vastly different from feature-driven approaches employed by saliency computation algorithms. The database comprises fixation patterns acquired using an eye-tracker, as subjects free-viewed images corresponding to many semantic categories such as faces (human and mammal), nudes and actions (look, read and shoot). The consistent presence of fixation clusters around specific image regions confirms that visual attention is not subjective, but is directed towards salient objects and object-interactions. We then show how the fixation clusters can be exploited for enhancing image understanding, by using our eye fixation database in an active image segmentation application. Apart from proposing a mechanism to automatically determine characteristic fixation seeds for segmentation, we show that the use of fixation seeds generated from multiple fixation clusters on the salient object can lead to a 10% improvement in segmentation performance over the state-of-the-art.

A DCT domain visible watermarking technique for images

The growth of computer networks has boosted the growth of the information technology sector to a greater extent. There is a trend to move from conventional libraries to digital libraries. In digital libraries images and text are made available through the Internet for scholarly research. At the same time care is taken to prevent the unauthorized use of the images commercially. In some cases the observer is encouraged to patronize the institution that owns the material. To satisfy both these needs simultaneously the owner needs to use visible watermarking. Visible watermarking is a type of digital watermarking used for protection of publicly available images. We describe a visible watermarking scheme that is applied into the host image in the DCT domain. A mathematical model has been developed for this purpose. We also propose a modification of the algorithm to make the watermark more robust.

Audio Based Event Detection for Multimedia Surveillance

With the increasing use of audio sensors in surveillance and monitoring applications, event detection using audio streams has emerged as an important research problem. This paper presents a hierarchical approach for audio based event detection for surveillance. The proposed approach first classifies a given audio frame into vocal and nonvocal events, and then performs further classification into normal and excited events. We model the events using a Gaussian mixture model and optimize the parameters for four different audio features ZCR, LPC, LPCC and LFCC. Experiments have been performed to evaluate the effectiveness of the features for detecting various normal and the excited state human activities. The results show that the proposed top-down event detection approach works significantly better than the single level approach

Progressive color visual cryptography

Visual cryptography is a powerful technique that combines the notions of perfect ciphers and secret sharing in cryptography with that of raster graphics. A binary image can be divided into shares that can be stacked together to approximately recover the original image. Unfortunately, it has not been used much primarily because the decryption process entails a severe degradation in image quality in terms of loss of resolution and contrast. Its usage is also hampered by the lack of proper techniques for handling gray-scale and color images. We develop a novel technique that enables visual cryptography of color as well as gray-scale images. With the use of halftoning and a novel microblock encoding scheme, the technique has a unique flexibility that enables a single encryption of a color image but enables three types of decryptions on the same ciphertext. The three different types of decryptions enable the recovery of the image of varying qualities. The physical transparency stacking type of decryption enables the recovery of the traditional visual cryptography quality image. An enhanced stacking technique enables the decryption into a halftone quality image. Finally, a computation-based decryption scheme makes the perfect recovery of the original image possible. Based on this basic scheme, we establish a progressive mechanism to share color images at multiple resolutions. We extract shares from each resolution layer to construct a hierarchical structure; the images of different resolutions can then be restored by stacking the different shared images together. Thus, our technique enables flexible decryption. We implement our technique and present results.

Depth matters: influence of depth cues on visual saliency

Most previous studies on visual saliency have only focused on static or dynamic 2D scenes. Since the human visual system has evolved predominantly in natural three dimensional environments, it is important to study whether and how depth information influences visual saliency. In this work, we first collect a large human eye fixation database compiled from a pool of 600 2D-vs-3D image pairs viewed by 80 subjects, where the depth information is directly provided by the Kinect camera and the eye tracking data are captured in both 2D and 3D free-viewing experiments. We then analyze the major discrepancies between 2D and 3D human fixation data of the same scenes, which are further abstracted and modeled as novel depth priors. Finally, we evaluate the performances of state-of-the-art saliency detection models over 3D images, and propose solutions to enhance their performances by integrating the depth priors.