Jiankang Wu

CORE: a content-based retrieval engine for multimedia information systems

Rapid advances in multimedia technology necessitate the development of a generic multimedia information system with a powerful retrieval engine for prototyping multimedia applications. We develop a content-based retrieval engine (CORE) that makes use of novel indexing techniques for multimedia object retrieval. We formalize the concepts related to multimedia information systems such as multimedia objects and content-based retrieval. We bring out the requirements and challenges of a multimedia information system. The architecture of CORE is described in detail along with the associated retrieval mechanisms and indexing techniques. Various modules developed for efficient retrieval are presented with some APIs. The efficacy of CORE is demonstrated in the development of two multimedia systems, a computer-aided facial image inference and retrieval (CAFIIR) system and a system for trademark archival and retrieval (STAR), which have been developed at the Institute of Systems Science (ISS). We expect that CORE will be useful for effective prototyping of other such multimedia applications.

Content-Based Retrieval for Trademark Registration

With ever increasing number of registered trademarks, the task of trademark office is becoming increasingly difficult to ensure the uniqueness of all trademarks registered. Trademarks are complex patterns consisting of various image and text patterns, called device-mark and word-in-mark respectively. Due to the diversity and complexity of image patterns occurring in trademarks, due to multi-lingual word-in-mark, there is no very successful computerized operating trademark registration system. We have tackled key technical issues: multiple feature extraction methods to capture the shape, similarity of multi-lingual word-in-mark, matching device mark interpretation using fuzzy thesaurus, and fusion of multiple feature measures for conflict trademark retrieval. A prototype System for Trademark Archival and Registration (STAR) has been developed. The initial test run has been conducted using 3000 trademarks, and the results have shown satisfaction to trademark officers and specialists.

Print signatures for document authentication

We present a novel solution for authenticating printed paper documents by utilizing the inherent non--repeatable randomness existing in the printing process. For a document printed by a laser-printer, we extract the unique features of the non--repeatable print content for each copy. The shape profiles of this content are used as the feature to represent the uniqueness of that particular printed copy. These features along with some important document content is then captured as the print signature. We present theoretical and experimental details on how to register as well as authenticate this print signature. The security analysis of this technique is also presented. We finally provide experimental results to demonstrate the feasibility of the proposed method.

Benchmarking Multimedia Databases

Multimedia technologies are being adopted both in the professional and commercial world with great enthusiasm. This has led to a significant interest in the research and development of multimedia databases. However, none of these efforts have really addressed the issues related to the benchmarking of multimedia databases. We analyze the problem of benchmarking multimedia databases in this paper and suggest a methodology.

Identifying faces using multiple retrievals

During a police investigation, officers often have to sort through hundreds of photographs to identify a suspect. To aid this task, we at the Institute of Systems Science developed and implemented a flexible database system that can retrieve faces using personal information, fuzzy and free-text descriptors, and classification trees.<<ETX>>

A robust approach to face and eyes detection from images with cluttered background

Automatic face location in complex scenes is extremely challenging in human face recognition systems. Further more, the facial features detection also plays an important role. The paper presents a scheme for robust face and eyes detection from an image. The scheme uses the Gaussian steerable filter to search and detect the facial feature (preattentive feature) roughly in an image. The face model is investigated to locate the whole face and facial features, such as eyes, nose and mouth. Here, multiple evidences are used in the face location and eyes detection. One important feature is the structural information of the face, i.e. facial components of certain structure. The other is the symmetry property of the face, here only the front face with certain pose variation is considered. It will reduce the computation greatly. For facial components detection, some image features and PCA features are used for verification from the candidates detected before. Experiments show that the algorithm is robust and fast.

STAR — A multimedia database system for trademark registration

To ensure the uniqueness of all trademarks registered is very important. With ever increasing number of registered trademarks, this is becoming increasingly difficult. A system for search and registration of trademarks is presented in this paper. Trademarks are complex patterns consisting of various image and text patterns, called devicemark and word-in-mark respectively. Traditionally, only text part has been used for search and retrieval of such patterns. This was largely due to the diversity and complexity of image patterns occurring in trademarks. The System for Trademark Archival and Registration (STAR) presented here, uses features based on image as well as text components of trademarks and brings out the conflicting trademarks for the consideration of trademark officer. Thus, it simplifies the task of trademark office to a great extent. A structural representation consisting of image, graphics, text, and phonetics has been proposed to handle the diversity and complexity of trademarks. Based on this structural representation, an object-orieted database schema, a sophisticated segmentation technique, a composite similarity measure for searching conflicting trademarks, and an indexing scheme have been developed. Initial results are very promising. More testing is in progress.

Inference and retrieval of facial images

Attempts have been made to extend SQL to work with multimedia databases. We are reserved on the representation ability of extended SQL to cope with the richness in content of multimedia data. In this paper we present an example of a multimedia database system, Computer Aided Facial Image Inference and Retrieval system (CAFIIR). The system stores and manages facial images and criminal records, providing necessary functions for crime identification. We would like to demonstrate some core techniques for multimedia database with CAFIIR system. Firstly, CAFIIR is a integrated system. Besides database management, there are image analysis, image composition, image aging, and report generation subsystems, providing means for problem solving. Secondly, the richness of multimedia data urges feature-based database for their management. CAFIIR is feature-based. A indexing mechanism,iconic index, has been proposed for indexing facial images using hierarchical self-organization neural network. The indexing method operates on complex feature measures and provides means for visual navigation. Thirdly, special retrieval methods for facial images have been developed, including visual browsing, similarity retrieval, free text retrieval and fuzzy retrieval.

Content-based representative frame extraction for digital video

We consider a content-based adaptive clustering algorithm (CBAC) to extract a fixed pre-determined number of representative frames to summarize a given digital video. In our algorithm, shot boundary detection is not needed. The inter-frame changes are compared globally for the extraction of representative frames. Small units of the video are dynamically clustered into two clusters based on the amount of change in content within the unit. One cluster, which has units of low content change, is designated for deletion and the other cluster, which has units of high content change, is for retention. The algorithm adaptively converges to the desired number of frames by deleting some redundant frames from each unit of the deletion cluster during every iteration.

Fuzzy content-based retrieval in image databases

Image data are inherently visual. The description of visual characteristics of images are imprecise. Fuzzy retrieval of images stored in a feature-based image database is a natural means to access the data. Unfortunately, to the authors knowledge, little work has been done on fuzzy image database models and fuzzy retrieval of feature-based image databases. In this paper, a fuzzy image database model and a concept of fuzzy space are proposed and fuzzy query processing in fuzzy space and fuzzy indexing on complex fuzzy vectors are described. An example image database, the computer-aided facial image inference and retrieval system (CAFIIR), is used for explanation throughout the paper.