Tomohiro Nakai

Use of affine invariants in locally likely arrangement hashing for camera-based document image retrieval

Camera-based document image retrieval is a task of searching document images from the database based on query images captured using digital cameras. For this task, it is required to solve the problem of “perspective distortion” of images,as well as to establish a way of matching document images efficiently. To solve these problems we have proposed a method called Locally Likely Arrangement Hashing (LLAH) which is characterized by both the use of a perspective invariant to cope with the distortion and the efficiency: LLAH only requires O(N) time where N is the number of feature points that describe the query image. In this paper, we introduce into LLAH an affine invariant instead of the perspective invariant so as to improve its adjustability. Experimental results show that the use of the affine invariant enables us to improve either the accuracy from 96.2% to 97.8%, or the retrieval time from 112 msec./query to 75 msec./query by selecting parameters of processing.

Real-Time Retrieval for Images of Documents in Various Languages Using a Web Camera

We propose a real-time retrieval method for document images in various languages. In this method, queries are images of documents captured by a web-camera. The document images corresponding to the queries are retrieved from the document image database in real time. Since we have already proposed a document image retrieval method for English documents, the proposed method is an extension for retrieval of documents in various languages. In the previous English document image retrieval method, only centroids of word regions are used as feature points. Therefore it cannot be applied to some languages including Japanese and Chinese due to no separation between words and periodic arrangements of characters. In the proposed method, additional features are introduced to realize real-time retrieval for document images in various languages.

Improvement of Retrieval Speed and Required Amount of Memory for Geometric Hashing by Combining Local Invariants

Thegeometrichashing(GH) is a well-knownmodel-basedobject recognition techniquewith goodpropertiesbothin retrievalspeedandrequiredamountof memory. However, it has a significant weak point; as the number of objects increases, both retrieval speed and required amount of memory increase in the cubic, fourth or higher order. Recently, a new technique “locally likely arrangement hashing (LLAH)” whose computational cost is a linear order has been proposed. The objective of the current paper is to reveal how LLAH improves the performance. By comparing GH and LLAH, we describe four primary factors of the performance improvement.

A Method of Annotation Extraction from Paper Documents Using Alignment Based on Local Arrangements of Feature Points

Annotations on paper documents include important information. We can exploit the information by extracting and analyzing annotations. In this paper, we propose a method of annotation extraction from paper documents. Unlike previous methods which limit colors or types of annotations to be extracted, the proposed method attempts to extract annotations by comparing a document image of an annotated document with its original document image for removing the limitations. The proposed method is characterized by fast matching and flexible subtraction of images both of which are essential to the annotation extraction by comparison. Experimental results have shown that color annotations can be extracted from color documents.

Capturing Digital Ink as Retrieving Fragments of Document Images

This paper presents a new method of capturing digital ink for pen-based computing. Current technologies such as tablets, ultrasonic and the Anoto pens rely on special mechanisms for locating the pen tip,which result in limiting the applicability.Our proposal is to ease this problem --- a camera pen that allows us to write on ordinary paper for capturing digital ink. A document image retrieval method called LLAH is tuned to locate the pen tip efficiently and accurately on the coordinates of a document only by capturing its tiny fragment.In this paper, we report some results on captured digital ink as well as to evaluate their quality.

Camera-based document image mosaicing using LLAH

In this paper we propose a mosaicing method of camera-captured document images. Since document images captured using digital cameras suffer from perspective distortion, their alignment is a diffcult task for previous methods. In the proposed method, correspondences of feature points are calculated using an image retrieval method LLAH. Document images are aligned using a perspective transformation parameter estimated from the correspondences. Since LLAH is invariant to perspective distortion, feature points can be matched without compensation of perspective distortion. Experimental results show that document images captured by a digital camera can be stitched using the proposed method.

Accuracy Improvement and Objective Evaluation of Annotation Extraction from Printed Documents

There is an approach of annotation extraction from printed documents in which annotations are extracted by comparing the image of an annotated document and its original document image. In one of the previous methods, the image of an original document is actually printed and scanned in order to reproduce image degradations of the image of the annotated document. However such a method lacks convenience since users have to use the same printer and scanner to obtain images of an annotated document and its original document. In this paper, we propose an improved annotation extraction method in which the image degradations are compensated by image processing. In the proposed method, the difference between original and annotated document images due to image degradations is reduced by not only removal of the degradations in the annotated document images but also reproduction of the degradations in the original document images. The proposed method consists of three steps of processing which are for dithering, for color change, and for local displacement. We also propose an objective evaluation of extracted annotations to compare the experimental results accurately. Experimental results of the proposed method have shown that the recall of extracted annotations was 80.94% and the precision was 85.59%.

Efficient Recognition of Planar Objects Based on Hashing of Keypoints - An Approach Towards Making the Physical World Clickable

This paper presents a method of planar object recognition for aiming at accessing information about objects by taking pictures of them. For this purpose efficiency of processing is the central issue because current state-of-the-art technologies with tree structures do not necessarily work well with a large amount of data represented as high dimensional vectors. To solve this problem, we employ hashing of keypoints extracted from images of objects. With the help of hash keys obtained as integers converted from the real valued vectors, keypoints are stored with object IDs and retrieved with no search process. Voting for object IDs is employed to determine a recognized object as the one with the largest vote. Experimental results show that the proposed method is at least 400 times faster than a brute-force method while 90% of objects were correctly recognized