Jaap Haitsma

A Review of Audio Fingerprinting

An audio fingerprint is a compact content-based signature that summarizes an audio recording. Audio Fingerprinting technologies have attracted attention since they allow the identification of audio independently of its format and without the need of meta-data or watermark embedding. Other uses of fingerprinting include: integrity verification, watermark support and content-based audio retrieval. The different approaches to fingerprinting have been described with different rationales and terminology: Pattern matching, Multimedia (Music) Information Retrieval or Cryptography (Robust Hashing). In this paper, we review different techniques describing its functional blocks as parts of a common, unified framework.

A review of algorithms for audio fingerprinting

An audio fingerprint is a content-based compact signature that summarizes an audio recording. Audio fingerprinting technologies have recently attracted attention since they allow the monitoring of audio independently of its format and without the need of meta-data or watermark embedding. The different approaches to fingerprinting are usually described with different rationales and terminology depending on the background: pattern matching, multimedia (music) information retrieval or cryptography (robust hashing). In this paper, we review different techniques mapping functional parts to blocks of a unified framework.

Video watermarking system for broadcast monitoring

This paper presents a video watermarking technology for broadcast monitoring. The technology has been developed at the Philips Research Laboratories in Eindhoven in the context of the European ESPRIT project VIVA (Visual Identity Verification Auditor). The aim of the VIVA project is to investigate and demonstrate a professional broadcast surveillance system. The key technology in the VIVA project is a new video watermarking technique by the name of JAWS (Just Another Watermarking System). The JAWS system has been developed such that the embedded watermarks (1) are invisible, (2) are robust with respect to all common processing steps in the broadcast transmission chain, (3) have a very low probability of false alarms, (4) have a large payload at high rate, and (5) allow for a low complexity and a real-time detection. In this paper we present the basic ingredients of the JAWS technology. We also briefly discuss the performance of JAWS with respect to the requirements of broadcast monitoring.

Feature Extraction and a Database Strategy for Video Fingerprinting

This paper presents the concept of video fingerprinting as a tool for video identification. As such, video fingerprinting is an important tool for persistent identification as proposed in MPEG-21. Applications range from video monitoring on broadcast channels to filtering on peer-to-peer networks to meta-data restoration in large digital libraries. We present considerations and a technique for (i) extracting essential perceptual features from moving image sequences and (ii) for identifying any sufficiently long unknown video segment by efficiently matching the fingerprint of the short segment with a large database of pre-computed fingerprints.

A robust image fingerprinting system using the Radon transform

With the ever-increasing use of multimedia contents through electronic commerce and on-line services, the problems associated with the protection of intellectual property, management of large database and indexation of content are becoming more prominent. Watermarking has been considered as efficient means to these problems. Although watermarking is a powerful tool, there are some issues with the use of it, such as the modification of the content and its security. With respect to this, identifying content itself based on its own features rather than watermarking can be an alternative solution to these problems. The aim of fingerprinting is to provide fast and reliable methods for content identification. In this paper, we present a new approach for image fingerprinting using the Radon transform to make the fingerprint robust against affine transformations. Since it is quite easy with modern computers to apply affine transformations to audio, image and video, there is an obvious necessity for affine transformation resilient fingerprinting. Experimental results show that the proposed fingerprints are highly robust against most signal processing transformations. Besides robustness, we also address other issues such as pairwise independence, database search efficiency and key dependence of the proposed method.

A watermarking scheme for digital cinema

In this paper a video watermarking scheme is presented, that is designed for the future digital cinema format, as it will be used on large projector screens in theatres. The watermark is designed such that it has minimal impact on the video quality, but still is detectable after capture with a handheld camera and conversion to, for instance, VHS, CD-Video or DVD format. In order to achieve the severe requirements, concerning visibility and robustness, the proposed watermarking system only exploits the temporal axis. A watermark is inserted by changing the mean of the luminance values of a frame according to the samples of the watermark. Watermark detection is performed by correlating the watermark sequence with extracted mean luminance values of a sequence of frames. A demonstrator, implementing the proposed algorithm, has been built which shows the aforementioned requirements can be met with the proposed scheme.

Audio watermarking for monitoring and copy protection

Based on existing technology used in image and video watermarking, we have developed a robust audio watermarking technique. The embedding algorithm operates in frequency domain, where the magnitudes of the Fourier coefficients are slightly modified. In the temporal domain, an additional scale parameter and gain function are necessary to refine the watermark and achieve perceptual transparency. Watermark detection relies on the Symmetrical Phase Only Matched Filtering (SPOMF) cross-correlation approach. Not only the presence of a watermark, but also its cyclic shift is detected. This shift supports a multi-bit payload for one particular watermark sequence. The watermarking technology proved to be very robust to a large number of signal processing “attacks” such as MP3 (64 kb/s), all-pass filtering, echo addition, time-scale modification, resampling, noise addition, etc. It is expected that this approach may contribute in a wide variety of existing (e.g. monitoring and copy protection) and future applications.

Efficient detection of a spatial spread-spectrum watermark in MPEG video streams

Watermarking is a technique for embedding imperceptible labels in audio/visual content. Important parameters for any watermarking scheme are imperceptibility, robustness, security, payload, error rates and complexity. In the past few years enormous progress has been made with respect to most of these parameters, except maybe for the latter one. We address this topic of complexity by presenting an efficient method for detecting a spatial video watermark directly in an MPEG video stream. The naive approach would be to do a full MPEG decoding and then apply spatial watermark detection. However, such a full decoding approach would imply large memory and processing resources. We show that, at the cost of a slight decrease of performance, large reductions in complexity are possible. More in particular we show how high speed MPEG parsing, high speed inverse DCT processing and motion compensation can be avoided. On a more abstract level, this paper would like to encourage others to give more prominence to embedding and detection complexity as an important parameter in the design of watermarking schemes.

Exploiting shift invariance to obtain a high payload in digital image watermarking

Robust, invisible watermarking of digital images or video very often has to satisfy a set of mutually conflicting requirements. These requirements include invisibility, robustness to image processing and geometric transformations, low false positive probabilities, high payload, fast detection and embedding, etc. In this paper we show how invariance to translations can be exploited to increase the payload. This is achieved by simultaneous embedding of several shifted watermark patterns, such that the information content is hidden in the relative shifts of the patterns. The principles of this are illustrated for the case of JAWS, a spatial domain watermark method developed by Philips. The method can easily be applied to other watermark methods which are able to detect shifted versions of watermarks.

Speed-change resistant audio fingerprinting using auto-correlation

At ISMIR 2002 and CBMI 2001, the authors presented a new approach to audio fingerprinting (Haitsma, J. and Kalker, T., Proc. Int. Conf. on Music Information Retrieval, p.107-15, 2002; Haitsma et al., Proc. Int. Workshop on Content-Based Multimedia Indexing, p.117-25, 2001). The proposed scheme, which we refer to as the streaming audio fingerprinting (SAF) system, allows a very efficient database lookup and is also very robust against many different audio processing steps, including low bit rate audio coding, noise addition and amplitude compression. However it is not inherently robust against large linear speed changes (i.e. speed changes larger than 2%) where both the pitch and the tempo change. This is a potential problem, because some radio stations speed up by a few percent. We discuss a modification of the originally proposed fingerprinting algorithm to make it robust against large linear speed changes. The proposed modification has negligible effect on other aspects, such as robustness and reliability.