Rafael Dueire Lins

Assessing sentence scoring techniques for extractive text summarization

Abstract Text summarization is the process of automatically creating a shorter version of one or more text documents. It is an important way of finding relevant information in large text libraries or in the Internet. Essentially, text summarization techniques are classified as Extractive and Abstractive. Extractive techniques perform text summarization by selecting sentences of documents according to some criteria. Abstractive summaries attempt to improve the coherence among sentences by eliminating redundancies and clarifying the contest of sentences. In terms of extractive summarization, sentence scoring is the technique most used for extractive text summarization. This paper describes and performs a quantitative and qualitative assessment of 15 algorithms for sentence scoring available in the literature. Three different datasets (News, Blogs and Article contexts) were evaluated. In addition, directions to improve the sentence extraction results obtained are suggested.

Cyclic reference counting with lazy mark-scan

this paper is much higher than the original one for cyclic reference counting with local mark-scan. More shared cells will now be claimed directly, without any need for mark-scan. The deletion of the last pointer to a shared cell will recycle it immediately, regardless of whether there is a reference to it on the queue. The queue will be left basically with pointers to cycles and pointers to green cells in the free-list or recycled. In this case again, our algorithm performs far better than the original one. In the best case, only one local mark-scan will be performed per cycle, instead of as many as the number of external references to a cycle, as before.

Cyclic reference counting with local mark-scan

The general idea of the algorithm is to perform a local mark-scan whenever a pointer to a shared structure is deleted. The algorithm works in three phases. In the first phase we scan the graph below the deleted pointer, rearranging counts due to internal references and marking the nodes as possible garbage. In phase two, the graph is rescanned and any subgraphs with external references are remarked as ordinary cells, and their counts reset. All other nodes are marked as garbage. Finally, in phase three all garbage cells are collected and returned to the free list

A fast orientation and skew detection algorithm for monochromatic document images

Very often in the digitization process, documents are either not placed with the correct orientation or are rotated of small angles in relation to the original image axis. These factors make more difficult the visualization of images by human users, increase the complexity of any sort of automatic image recognition, degrade the performance of OCR tools, increase the space needed for image storage, etc. This paper presents a fast algorithm for orientation and skew detection for complex monochromatic document images, which is capable of detecting any document rotation at a high precision.

A multi-document summarization system based on statistics and linguistic treatment

The massive quantity of data available today in the Internet has reached such a huge volume that it has become humanly unfeasible to efficiently sieve useful information from it. One solution to this problem is offered by using text summarization techniques. Text summarization, the process of automatically creating a shorter version of one or more text documents, is an important way of finding relevant information in large text libraries or in the Internet. This paper presents a multi-document summarization system that concisely extracts the main aspects of a set of documents, trying to avoid the typical problems of this type of summarization: information redundancy and diversity. Such a purpose is achieved through a new sentence clustering algorithm based on a graph model that makes use of statistic similarities and linguistic treatment. The DUC 2002 dataset was used to assess the performance of the proposed system, surpassing DUC competitors by a 50% margin of f-measure, in the best case.

Automatic language identification of written texts

Language identification is one of the search keys of most widespread use in the Internet. This article describes efficient and easily extensible solutions to the problem of identifying the language of written texts based on closed grammatical classes. An identification tool was developed for recognizing texts written in Portuguese, Spanish, French and English.

An environment for processing images of historical documents

This paper presents an environment for acquisition, filtering, compression and storage of images of historical documents.

Binarizing and filtering historical documents with back-to-front interference

This paper presents a new segmentation-based method for generating high-quality monochromatic images of historical documents. The proposed segmentation algorithm is based on the entropy of the histogram of the image. The algorithm eliminates back-to-front interference in documents written on both sides on translucent paper.

Efficient Removal of Noisy Borders of Monochromatic Documents

Very often the digitalization process using automatically fed production line scanners yields monochromatic images framed by a noisy border. This paper presents a pre-processing scheme based on sub sampling which speeds up the border removal process. The technique introduced was tested on over 20,000 images and provided same quality images than the best algorithm in the literature and amongst commercial tools with an average speed-up around 50%.

Generation of images of historical documents by composition

This paper describes a system for efficient storage, indexing and network transmission of images of historical documents. The documents are first decomposed into their features such as paper texture, colours, typewritten parts, pictures, etc. Document retrieval forces the re-assembling of the document, synthetising an image visually close to the original document. The information needed to build the final image occupies, in average, 2 Kbytes performing a very efficient compression scheme.