Yingmin Tang

Text Detection in Natural Images Using Localized Stroke Width Transform

How to effectively and efficiently detect texts in natural scene images is a challenging problem. This paper presents a novel text detection method using localized stroke width transform. Due to the utilization of an adaptive image binarization approach and the implementation of stroke width transform in local regions, our method markedly reduces the demand of contrast between texts and backgrounds, and becomes considerably robust against edge detection results. Experiments on the dataset of ICDAR 2013 robust reading competition demonstrate that the proposed method outperforms other state-of-the-art approaches in the application of text detection in natural scene images.

Skeleton-guided vectorization of Chinese calligraphy images

How to automatically generate compact and high-quality vectorization for Chinese calligraphy images is a challenging problem, since these images usually suffer from noisy contours and discontinuous strokes. In this paper, we propose a skeleton guided approach to vectorize Chinese calligraphy images. Since the skeleton reflects the writing trace and it is less influenced by contour noises, our method could extract the important writing style from the noisy contours. Specifically, in our method, the calligraphy image is first preprocessed by binarization and denoising. Then salient contour points are detected by a novel algorithm. Afterwards, under the guidance of skeleton information, the salient points are classified into corner points and joint points. Finally, a dynamic curve fitting procedure is applied to generate the vectorization result. Experimental results demonstrate that our skeleton-guided approach could automatically distinguish tiny features from contour noises and thus obtains more visually satisfactory performance compared to other existing methods.

Automatic Correspondence Finding for Chinese Characters Using Graph Matching

Automatically establishing correspondence between Chinese characters is a challenging task. In this paper, we propose a novel method to solve this problem. Given two Chinese characters, we first extract and properly prune the skeleton of each character to get the key points and the connectivity relations of these points. Then, the similarity between each pair of key points is calculated via the comparison of their local features. Afterwards, a set of edges are constructed by considering both the connectivity relations and k nearest neighbors (k-nn) of each point. Finally, correspondence between two characters is established by applying a guided graph matching algorithm. Experimental results demonstrate the effectiveness of our method for the correspondence problem of Chinese characters in both printing and handwritten styles. Moreover, we also show that our method can be utilized to automatically extract strokes from Chinese characters.

DCFont: an end-to-end deep chinese font generation system

Building a complete personalized Chinese font library for an ordinary person is a tough task due to the existence of huge amounts of characters with complicated structures. Yet, existing automatic font generation methods still have many drawbacks. To address the problem, this paper proposes an end-to-end learning system, DCFont, to automatically generate the whole GB2312 font library that consists of 6763 Chinese characters from a small number (e.g., 775) of characters written by the user. Our system has two major advantages. On the one hand, the system works in an end-to-end manner, which means that human interventions during offline training and online generating periods are not required. On the other hand, a novel deep neural network architecture is designed to solve the font feature reconstruction and handwriting synthesis problems through adversarial training, which requires fewer input data but obtains more realistic and high-quality synthesis results compared to other deep learning based approaches. Experimental results verify the superiority of our method against the state of the art.

Content-independent font recognition on a single Chinese character using sparse representation

Font recognition on a single Chinese character is a challenging task especially when the identity of the character is unknown and the number of possible font types is huge. In this paper, we propose a novel method using multi-scale sparse representation to solve the problem of large-scale font recognition on a single unknown Chinese character. Specifically, we first apply a saliency-based sampling approach, which exploits the saliency information of character contours, to segment local patches in multiple scales from salient regions. Then, corresponding local descriptors are extracted by implementing Sobel and Prewitt operators in 4 directions. After encoding the local descriptors into sparse codes, max pooling and spatial pyramid matching are employed to pool them into a sparse representation. Finally, a multi-scale sparse representation is obtained by concatenating three sparse representations which respectively correspond to three particular scales of local patches, and then the linear SVM classifier is utilized for font classification. Experiments performed on a large-scale database consisting of Chinese character images in 160 fonts show that our method achieves significantly better performance compared to the state of the art. Moreover, we also carry out experiments on a subset of the database to demonstrate the effectiveness of our saliency-based sampling approach and the proposed Sobel-Prewitt feature.

Non-rigid point set registration for Chinese characters using structure-guided coherent point drift

This paper proposes a non-rigid point set registration method called Structure-Guided Coherent Point Drift (SGCPD). The key idea of our method is to utilize structural information and combine the global and local point registrations together to improve the original Coherent Point Drift (CPD) algorithm. Specifically, given two point sets, we first align them using the CPD method with Localized Operator (CPDLO). Then we divide the target point set into several subsets and apply CPDLO to each subset. Finally, we implement the above two procedures until convergence. In this manner, more detailed information can be well exploited and thus higher registration accuracy can be achieved. Experimental results demonstrate that our method outperforms the original CPD approach on both point registration accuracy and skeleton decomposition accuracy for Chinese characters.

Font Recognition in Natural Images via Transfer Learning

Font recognition is an important and challenging problem in areas of Document Analysis, Pattern Recognition and Computer Vision. In this paper, we try to handle a tougher task that aims to accurately recognize the font styles of texts in natural images by proposing a novel method based on deep learning and transfer learning. Major contributions of this paper are threefold: First, we develop a fast and scalable system to synthesize huge amounts of natural images containing texts in various fonts and styles, which are then utilized to train the deep neural network for font recognition. Second, we design a transfer learning scheme to alleviate the domain mismatch between synthetic and real-world text images. Thus, large numbers of unlabeled text images can be adopted to markedly enhance the discrimination and robustness of our font classifier. Third, we build a benchmarking database which consists of numerous labeled natural images containing Chinese characters in 48 fonts. As far as we know, it is the first publicly-available dataset for font recognition of Chinese characters in natural images.

An Automatic Stroke Extraction Method using Manifold Learning

Stroke extraction is one of the most important tasks in areas of computer graphics and document analysis. So far, data-driven methods are believed to perform relatively well, which use the pre-processed characters as templates. However, how to accurately extract strokes of characters is still a tough and challenging task because there are various styles of characters, which may vary a lot from the template character. To solve this problem, we build a font skeleton manifold in which we can always find a most similar character as a template by traversing the locations in the manifold. Because of the similar structure and font style, the point set registration of the template character with the target character would be much more effective and accurate. Experimental results on characters in both printing style and handwriting style reveal that our method using manifold learning has a better performance in the application of stroke extraction for Chinese characters.

Structure-Aware Image Resizing for Chinese Characters

This paper presents a structure-aware resizing method for Chinese character images. Compared to other image resizing approaches, the proposed method is able to preserve important features such as the width, orientation and trajectory of each stroke for a given Chinese character. The key idea of our method is to first automatically decompose the character image into strokes, and then separately resize those strokes naturally using a modified linear blend skinning approach and as-rigid-as-possible shape interpolation under the guidance of structure information. Experimental results not only verify the superiority of our method compared to the state of the art but also demonstrate its effectiveness in several real applications.

FlexiFont: a flexible system to generate personal font libraries

This paper proposes FlexiFont, a system designed to generate personal font libraries from the camera-captured character images. Compared with existing methods, our system is able to process most kinds of languages and the generated font libraries can be extended by adding new characters based on the users requirement. Moreover, digital cameras instead of scanners are chosen as the input devices, so that it is more convenient for common people to use the system. First of all, the users should choose a default template or define their own templates, then write the characters on the printed templates according to the certain instructions. After the users upload the photos of the templates with written characters, the system will automatically correct the perspective and split the whole photo into a set of individual character images. As the final step, FlexiFont will denoise, vectorize, and normalize each character image before storing it into a TrueType file. Experimental results demonstrate the robustness and efficiency of our system.