Hiroaki Takebe

A Multi-Stage Strategy to Perspective Rectification for Mobile Phone Camera-Based Document Images

Document images captured by a mobile phone camera often have perspective distortions. Efficiency and accuracy are two important issues in designing a rectification system for such perspective documents. In this paper, we propose a new perspective rectification system based on vanishing point detection. This system achieves both the desired efficiency and accuracy using a multi-stage strategy: at the first stage, document boundaries and straight lines are used to compute vanishing points; at the second stage, text baselines and block aligns are utilized; and at the last stage, character tilt orientations are voted for the vertical vanishing point. A profit function is introduced to evaluate the reliability of detected vanishing points at each stage. If vanishing points at one stage are reliable, then rectification is ended at that stage. Otherwise, our method continues to seek more reliable vanishing points in the next stage. We have tested this method with more than 400 images including paper documents, signboards and posters. The image acceptance rate is more than 98.5% with an average speed of only about 60 ms.

Slide identification for lecture movies by matching characters and images

Slide identification is very important when creating e-Learning materials as it detects slides being changed during lecture movies. Simply detecting the change would not be enough for e-Learning purposes. Because, which slide is now displayed in the frame is also important for creating e-Learning materials. A matching technique combined with a presentation file containing answer information is very useful in identifying slides in a movie frame. We propose two methods for slide identification in this paper. The first is character-based, which uses the relationship between the character code and its coordinates. The other is image-based, which uses normalized correlation and dynamic programming. We used actual movies to evaluate the performance of these methods, both independently and in combination, and the experimental results revealed that they are very effective in identifying slides in lecture movies.

Highly accurate retrieval method of Japanese document images through a combination of morphological analysis and OCR

We have developed a method that allows Japanese document images to be retrieved more accurately by using OCR character candidate information and a conventional plain text search engine. In this method, the document image is first recognized by normal OCR to produce text. Keyword areas are then estimated from the normal OCR produced text through morphological analysis. A lattice of candidate- character codes is extracted from these areas, and then character strings are extracted from the lattice using a word-matching method in noun areas and a K-th DP-matching method in undefined word areas. Finally, these extracted character strings are added to the normal OCR produced text to improve document retrieval accuracy when u sing a conventional plain text search engine. Experimental results from searches of 49 OHP sheet images revealed that our method has a high recall rate of 98.2%, compared to 90.3% with a conventional method using only normal OCR produced text, while requiring about the same processing time as normal OCR.

Separation of overlapped color planes for document images

Color plane separation is very useful in processing color document images. Many reported methods take it as a multi-class classification problem and work not well in overlapped color regions. This paper proposed a simple but effective linear projection based method for separating overlapped color planes. The separation task is taken as a probability problem, i.e., in the output plane, target color should have high response and the other colors should have low response, or vice versa. Furthermore, it assumes that the number of foreground colors is low, typically one to four, and overlapped areas contain mixed colors instead of opaque covering. Experimental results demonstrate the effectiveness and flexibility of our method.

Occluded text restoration and recognition

Text occlusion is among the most intractable obstacles for OCR engines. A typical example in document images is visible watermark characters, which are often occluded by foreground contents. This paper proposes a solution by restoring watermark characters before recognition. The text restoration process consists a core module as patch-based restoration method, which reconstructs the missing areas by referring to similar patches from undamaged areas. The filling sequence is in a order based on the structure complexity inside each patch, which helps to suppress reconstruction error propagation. Furthermore, the patch size is adaptively selected based on the local character stroke width. Experiments show that the proposed method produces good restoration quality and effectively improves the recognition rate of the following OCR process. Furthermore, the algorithm is optimized based on statistical analysis model and the processing time meets the real-time responding requirement.

A Learning Pseudo Bayes Discriminant Method Based on Difference Distribution of Feature Vectors

We developed a learning pseudo Bayes discriminant method, that dynamically adapts a pseudo Bayes discriminant function to a font and image degradation condition present in a text. In this method, the characteristics of character pattern deformations are expressed as a statistic of a difference distribution, and information represented by the difference distribution is integrated into the pseudo Bayes discriminant function. The formulation of integrating the difference distribution into the pseudo Bayes discriminant function results in that a covariance matrix of each category is adjusted based on the difference distribution. We evaluated the proposed method on multifont texts and degraded texts such as compressed color images and faxed copies. We found that the recognition accuracy of our method for the evaluated texts was much higher than that of conventional methods.

Character string extraction from newspaper headlines with a background design by recognizing a combination of connected components

In this paper we propose a new method of extracting a character string from images with a background design. In Japanese newspaper headlines, it is common for character components to be placed independent of background components. In view of this, we represent a character string candidate as a consistent combination of connected components, and we calculate its character string resemblance value. In this case, a character string resemblance value of a combination of connected components depends upon its character recognition result and the area of the rectangular area occupied by it. We then extract the combination of connected components that has the maximum character string resemblance value. We applied this method to 142 headline images. The results show that the method accurately extracted a character string from various kinds of images with a background design and the method has a favorable processing speed.

Precise and Fast Form Identification Method by Using Adaptive Base Lines for Matching

Conventional form identification methods have been based on the normalization of an input image. So, if the base for normalization is different from that of the true model, it is difficult to identify its form. In this paper, we propose a form identification method, which prevents the difference from spreading throughout the process. In the method, the local ruled line structures are analyzed exhaustively by varying a pair of base lines of an input image and a model. The process is realized efficiently by generating the correspondence possibilities between ruled lines, and grouping these possibilities. We registered 100 models with a dictionary, and experimented on form identification under the various conditions. The result shows that the method has high accuracy and practical processing speed.

Trinary Image Mosaicing Based Watermark String Detection

Watermark string detection is very useful for document security protection. In [2], we proposed an image based document watermark detection system. In this paper, two major modifications are made based on the previous system.First, a trinary image mosaicing algorithm is proposed to merge the broken watermark string images into a high quality mosaic image. Second, we relax the conditions in the Maximum Clique based keyword detection algorithm.The new algorithm can handle watermark string with variant spacing. Therefore, not only English keywords, but also Japanese/Chinese keywords can be processed. The Experimental results show the effectiveness of our algorithm.

Logical Structure Analysis for Form Images with Arbitrary Layout by Belief Propagation

A new method for analyzing the specific logical structure of forms with unknown layout is proposed. This method uses both the target form image and a generic logical structure as inputs, and models two types of relationships probabilistically: that between strings and logical components, and that between neighboring strings having different logical components. This modeling approach allows strings to be assigned to logical components softly but robustly, and allows the use of an intuitive Bayesian probability network similar to the generic logical structure. Based on this probability network model, strings corresponding to logical components can be determined by belief propagation. This method is demonstrated to be effective by conducting tests on three types of forms.