Martin Lettner

Spatial and Spectral Based Segmentation of Text in Multispectral Images of Ancient Documents

In this paper we propose a character segmentation method for multispectral images of ancient documents. Due to the low quality of the images the main idea of this study is to combine the multispectral behavior and contextual spatial information. Therefore we utilize a Markov Random Field model using the spectral information of the images and stroke properties to include spatial dependencies of the characters. Since the stroke properties and the Gaussian parameters for the imaging model are evaluated automatically the proposed segmentation method requires no training phase. We compared the method to state of the art character segmentation methods and demonstrate the effectiveness of combining spectral and spatial features for the segmentation of characters in multispectral images.

Higher order MRF for foreground-background separation in multi-spectral images of historical manuscripts

Multi-spectral imaging for the analysis and preservation of ancient documents has gained high attention in recent years. While readability enhancement is based on the multi-spectral image corpus, foreground-background separation still relies mainly on gray level or color images. In this paper we propose a foreground-background separation algorithm designed for multi-spectral images. The main contribution is the simultaneously utilization of spectral and spatial features. While spectral features incorporate the spectral components of the multi-spectral images, the spatial features are based on stroke properties. Higher order Markov Random Fields enables an efficient way to combine both features. To solve higher order energy functions, we introduce a new message update rule in the well known belief propagation algorithm based on a higher order potential function.

Registration of multi-spectral manuscript images

Two medieval Slavonic manuscripts are recorded, investigated and analyzed by philologists in collaboration with computer scientists. The aim of the project is to develop algorithms that support the philologists by automatically deriving the description and restoration of the scripts. The parchment partially contains two scripts, where the first script was erased. In order to enhance the erased script, the manuscript pages are imaged in seven bands between 330 and 1000 nm. A registration, aligning the resultant images, is necessary so that further image processing algorithms can combine the information gained by the different spectral bands. Therefore, the images are coarsely aligned using rotationally invariant features and an affine transformation. Afterwards, the similarity of the different images is computed by means of the normalized cross correlation. Finally, the images are accurately mapped to each other by the local weighted mean transformation. The algorithms used for the registration and preliminary results are presented in this paper.

Contrast Enhancement in Multispectral Images by Emphasizing Text Regions

This paper deals with the enhancement of the readability in historic texts written on parchment. Due to mold, air, humidity, water, etc. parchment and text are partially damaged and consequently hard to read. In order to enhance the readability of the text, the manuscript pages are imaged in different spectral bands ranging from 360 to 1000 nm. The readability enhancement is based on a spectral and spatial analysis of the multivariate image data by multivariate spatial correlation. The main advantage of the method is that especially the text regions are enhanced which is provided by generating a mask image. This mask is based on the automatic reconstruction of the ruling scheme of the text pages. The method is tested on two medieval Slavonic manuscripts written on parchment.

Texture analysis for stroke classification in infrared reflectogramms

The recognition of painted strokes is an important step in analyzing underdrawings in infrared reflectogramms. But even for art experts, it is difficult to recognize all drawing tools and materials used for the creation of the strokes. Thus the use of computer-aided imaging technologies brings a new and objective analysis and assists the art experts. This work proposes a method to recognize strokes drawn by different drawing tools and materials. The method uses texture analysis algorithms performing along the drawing trace to distinguish between different types of strokes. The benefit of this method is the increased content of textural information within the stroke and simultaneously in the border region. We tested our algorithms on a set of six different types of strokes: 3 classes of fluid and 3 classes of dry drawing materials.

Estimating the original drawing trace of painted strokes

Pencil drawings like portraits or landscapes comprise dozens of strokes. The segmentation and identification of individual strokes is an interesting question in analyzing the drawings since it allows art historians to analyze the development of the stroke formations in the picture in more detail. In this study we are going to identify individual strokes in stroke formations and to reconstruct the original drawing trace of the artist. The method is based on a thinning algorithm and a following analysis of the accrued skeleton. In order to detect the original stroke and the natural drawing trace we use the curvilinearity information of the thinned sub-strokes. A sub-stroke runs from either a real end point to a crossing point, or between two crossing points. The selection of corresponding strokes in crossing points is based on the angle at the end points of the sub-strokes. The individual strokes drawn through are represented by a one pixel wide line which approximates the original drawing trace of the artist by a cubic B-spline. The whole process is parameter free: we use the automatic calculated stroke width for the skeleton pruning process, for the calculation of the angles at the sub-stroke endings and as the distance for the spline control points.

Combining Spectral and Spatial Features for Robust Foreground-Background Separation

Foreground-background separation in multispectral images of damaged manuscripts can benefit from both, spectral and spatial information. Therefore, we incorporate a Markov Random Field which provides a powerful tool to combine both features simultaneously. Higher order models enable the inclusion of spatial constraints based on stroke characteristics. We apply belief propagation for inference and include the higher order potentials by upgrading the message update. The proposed segmentation method requires no training and is independent of script, size, and style of characters. We will demonstrate the robust performance on a set of degraded documents and on synthetic images.

Texture and Profile Features for Drawing Media Recognition in Underdrawings

In this study we analyze texture and profile features of painted strokes in order to identify the drawing media used for sketching underdrawings. Underdrawings are preliminary drawings on the panel prepared for paintings and are unseen in the finished work. Cameras working in the near infrared range allow the visualization of underdrawings. Due to the tiny width of the strokes we perform an alignment of the feature extraction windows in order to obtain a major content of the stroke texture. The method is tested on strokes applied on test panels and underdrawing strokes in IR images of medieval paintings

Image Acquisition & Processing Routines for Damaged Manuscripts

This paper presents an overview of data acquisition and processing procedures of an interdisciplinary project of philologists and image processing experts aiming at the decipherment and reconstruction of damaged manuscripts. The digital raw image data was acquired via multi-spectral imaging. As a preparatory step we developed a method of foreground-background separation (binarisation) especially designed for multi-spectral images of degraded documents. On the basis of the binarised images further applications were developed: an automatic character decomposition and primitive extraction dissects the scriptural elements into analysable pieces that are necessary for palaeographic and graphemic analyses, writing tool recognition, text restoration, and optical character recognition. The results of the relevant procedures can be stored and interrogated in a database application. Furthermore, a semi-automatic page layout analysis provides codicological information on latent page contents (script, ruling, decorations).