Shangxuan Tian

Text Flow: A Unified Text Detection System in Natural Scene Images

The prevalent scene text detection approach follows four sequential steps comprising character candidate detection, false character candidate removal, text line extraction, and text line verification. However, errors occur and accumulate throughout each of these sequential steps which often lead to low detection performance. To address these issues, we propose a unified scene text detection system, namely Text Flow, by utilizing the minimum cost (min-cost) flow network model. With character candidates detected by cascade boosting, the min-cost flow network model integrates the last three sequential steps into a single process which solves the error accumulation problem at both character level and text line level effectively. The proposed technique has been tested on three public datasets, i.e, ICDAR2011 dataset, ICDAR2013 dataset and a multilingual dataset and it outperforms the state-of-the-art methods on all three datasets with much higher recall and F-score. The good performance on the multilingual dataset shows that the proposed technique can be used for the detection of texts in different languages.

Recognizing Text with Perspective Distortion in Natural Scenes

This paper presents an approach to text recognition in natural scene images. Unlike most existing works which assume that texts are horizontal and frontal parallel to the image plane, our method is able to recognize perspective texts of arbitrary orientations. For individual character recognition, we adopt a bag-of-key points approach, in which Scale Invariant Feature Transform (SIFT) descriptors are extracted densely and quantized using a pre-trained vocabulary. Following [1, 2], the context information is utilized through lexicons. We formulate word recognition as finding the optimal alignment between the set of characters and the list of lexicon words. Furthermore, we introduce a new dataset called StreetViewText-Perspective, which contains texts in street images with a great variety of viewpoints. Experimental results on public datasets and the proposed dataset show that our method significantly outperforms the state-of-the-art on perspective texts of arbitrary orientations.

Scene Text Recognition Using Co-occurrence of Histogram of Oriented Gradients

Scene text recognition is a fundamental step in End-to-End applications where traditional optical character recognition (OCR) systems often fail to produce satisfactory results. This paper proposes a technique that uses co-occurrence histogram of oriented gradients (Co-HOG) to recognize the text in scenes. Compared with histogram of oriented gradients (HOG), Co-HOG is a more powerful tool that captures spatial distribution of neighboring orientation pairs instead of just a single gradient orientation. At the same time, it is more efficient compared with HOG and therefore more suitable for real-time applications. The proposed scene text recognition technique is evaluated on ICDAR2003 character dataset and Street View Text (SVT) dataset. Experiments show that the Co-HOG based technique clearly outperforms state-of-the-art techniques that use HOG, Scale Invariant Feature Transform (SIFT), and Maximally Stable Extremal Regions (MSER).

Multilingual scene character recognition with co-occurrence of histogram of oriented gradients

Automatic machine reading of texts in scenes is largely restricted by the poor character recognition accuracy. In this paper, we extend the Histogram of Oriented Gradient (HOG) and propose two new feature descriptors: Co-occurrence HOG (Co-HOG) and Convolutional Co-HOG (ConvCo-HOG) for accurate recognition of scene texts of different languages. Compared with HOG which counts orientation frequency of each single pixel, the Co-HOG encodes more spatial contextual information by capturing the co-occurrence of orientation pairs of neighboring pixels. Additionally, ConvCo-HOG exhaustively extracts Co-HOG features from every possible image patches within a character image for more spatial information. The two features have been evaluated extensively on five scene character datasets of three different languages including three sets in English, one set in Chinese and one set in Bengali. Experiments show that the proposed techniques provide superior scene character recognition accuracy and are capable of recognizing scene texts of different scripts and languages. HighlightsIntroduced powerful features Co-HOG and ConvCo-HOG for scene character recognition.Designed a new offset based strategy for dimension reduction of above features.Developed two new scene character datasets for Chinese and Bengali scripts.Extensive simulations on 5 datasets of 3 scripts show the efficiency of the approach.

Character Recognition in Natural Scenes Using Convolutional Co-occurrence HOG

Recognition of characters in natural images is a challenging task due to the complex background, variations of text size and perspective distortion, etc. Traditional optical character recognition (OCR) engine cannot perform well on those unconstrained text images. A novel technique is proposed in this paper that makes use of convolutional cooccurrence histogram of oriented gradient (ConvCoHOG), which is more robust and discriminative than both the histogram of oriented gradient (HOG) and the co-occurrence histogram of oriented gradients (CoHOG). In the proposed technique, a more informative feature is constructed by exhaustively extracting features from every possible image patches within character images. Experiments on two public datasets including the ICDAr 2003 Robust Reading character dataset and the Street View Text (SVT) dataset, show that our proposed character recognition technique obtains superior performance compared with state-of-the-art techniques.

Scene Text Segmentation with Multi-level Maximally Stable Extremal Regions

The segmentation of scene text from the image background has shown great importance in scene text recognition. In this paper, we propose a multi-level MSER technology that identifies the best-quality text candidates from a set of stable regions that are extracted from different color channel images. In order to identify the best-quality text candidates, a segmentation score is defined which exploits four measures to evaluate the text probability of each stable region including: 1) Stroke width that measures the small stroke width variation of the text, 2) Boundary curvature that measures the smoothness of the stable region boundary, 3) Character confidence that measures the likelihood of a stable region being text based on a pre-trained support vector classifier, 4) Color constancy that measures the global color consistency of each selected text candidate. Finally, the MSERs with the best segmentation score from each channel are combined to form the final segmentation. The proposed method is evaluated on the ICDAR2003 and SVT datasets and experiments show that it outperforms both popular document image binarization methods and state of the art scene text segmentation methods.

Character shape restoration system through medial axis points in video

Shape restoration for characters in video is challenging because natural scene characters usually suffer from low resolution, complex background and perspective distortion. In this paper, we propose histogram gradient division and reverse gradient orientation in a new way to select Text Pixel Candidates (TPC) for a given input character. We apply a ring radius transform on TPC in different directions, namely, horizontal, vertical, principal and secondary diagonals in a TPC image to obtain respective radius maps, where each pixel is assigned a value that is the radius to the nearest TPC. This helps in finding Medial Axis Points (MAP) by searching for the maximum radius values from their neighborhoods in a radius image. The union of all the medial axis points obtained from the respective directions at each location is considered as Candidate Medial Axis Points (CMAP) of the character. Then color difference and k-means clustering are proposed to eliminate false CMAP, which outputs Potential Medial Axis Points (PMAP). We finally propose a novel way to restore the shape of the character from the PMAP. The method is tested on a video dataset and the benchmark ICDAR 2013 dataset to show its effectiveness for complex background and low resolution. Experimental results show that the proposed method is superior to the existing methods in terms of shape restoration error and recognition rate.

Scene Character Reconstruction through Medial Axis

Character shape reconstruction for the scene character is challenging and interesting because scene character usually suffers from uneven illumination, complex background, perspective distortion. To address such ill conditions, we propose to utilize Histogram Gradient Division (HGD) and Reverse Gradient Orientation (RGO) to select Candidate Text Pixels (CTPs) for a given input character. Ring Radius Transform is applied on each pixel in a CTP image to obtain radius map where each pixel is assigned a value which is the radius to the nearest CTP. Candidate medial axis pixels are those having maximum radius values in their neighborhoods. We find such pixels on horizontal, vertical, principal diagonal and secondary diagonal directions to determine the respective medial axis pixels. The union of all medial axis pixels at each pixel location is considered as a candidate medial axis pixel of the character. Then color difference and k-means clustering are employed to eliminate false candidate medial axis. The potential medial axis values are used to reconstruct the shape of the character. The method is tested on 1025 characters of complex foreground and background from ICDAR 2003 dataset in terms of shape reconstruction and recognition rate. Experimental results demonstrate the effectiveness of our proposed method for complex foreground and background characters in terms of character recognition rate and reconstruction error.

Using pyramid of histogram of oriented gradients on natural scene text recognition

Because of the unconstrained environment of scene text, traditional Optical Character Recognition (OCR) engines fail to achieve satisfactory results. In this paper, we propose a new technique which employs first order Histogram of Oriented Gradient (HOG) through a spatial pyramid. The spatial pyramid can encode the relative spatial layout of the character parts while HOG can only include the local image shape without spatial relation. A feature descriptor combining these two can extracts more useful information from the image for text recognition. Chi-square kernel based Support Vector Machine is employed for classification based on the proposed feature descriptors. The method is tested on three public datasets, namely ICDAR2003 robust reading dataset, Street View Text (SVT) dataset and IIIT 5K-word dataset. The results on these dataset are comparable with the state-of-the-art methods.

Robust text segmentation using graph cut

Text segmentation provides important clues for the accurate identification of character locations and the analysis of character properties such as shape estimation and texture synthesis. In this paper, we propose a robust text segmentation method that employs Markov Random Field (MRF) and use graph cut algorithms to solve the energy minimization problem. To effectively select accurate seeds to boost the text segmentation performance, stroke feature transform is adopted to robustly identify text seeds and text edges. Background seeds are obtained near the text edges in order to well preserve the text boundaries. The energy functions are defined as an MRF consisting of data energy and smoothness energy which can be efficiently solved by graph cut algorithms. One distinctive property of the proposed technique is that it can identify more distinctive seeds so that only one cut is needed to well separate the text regions from the background, hence much faster than the existing iterative graph cut approach. Experiments on ICDAR 2003 and ICDAR 2011 datasets show that the proposed technique obtains superior performance on both pixel level and atom level segmentation.