Anna Kazantseva

Summarizing short stories

We present an approach to the automatic creation of extractive summaries of literary short stories. The summaries are produced with a specific objective in mind: to help a reader decide whether she would be interested in reading the complete story. To this end, the summaries give the user relevant information about the setting of the story without revealing its plot. The system relies on assorted surface indicators about clauses in the short story, the most important of which are those related to the aspectual type of a clause and to the main entities in a story. Fifteen judges evaluated the summaries on a number of extrinsic and intrinsic measures. The outcome of this evaluation suggests that the summaries are helpful in achieving the original objective.

Getting emotional about news summarization

News is not simply a straight re-telling of events, but rather an interpretation of those events by a reporter, whose feelings and opinions can often become part of the story itself. Research on automatic summarization of news articles has thus far focused on facts rather than emotions, but perhaps emotions can be significant in news stories too. This article describes research done at the University of Ottawa to create an emotion-aware summarization system, which participated in the Text Analysis Conference last year. We have established that increasing the number of emotional words could help ranking sentences to be selected for the summary, but there was no overall improvement in the final system. Although this experiment did not improve news summarization as evaluated by a variety of standard scoring techniques, it was successful at generating summaries with more emotional words while maintaining the overall quality of the summary.

An approach to summarizing short stories

This paper describes a system that produces extractive summaries of short works of literary fiction. The ultimate purpose of produced summaries is defined as helping a reader to determine whether she would be interested in reading a particular story. To this end, the summary aims to provide a reader with an idea about the settings of a story (such as characters, time and place) without revealing the plot. The approach presented here relies heavily on the notion of aspect. Preliminary results show an improvement over two naive baselines: a lead baseline and a more sophisticated variant of it. Although modest, the results suggest that using aspectual information may be of help when summarizing fiction. A more thorough evaluation involving human judges is under way.

NRC Russian-English Machine Translation System for WMT 2016

We describe the statistical machine translation system developed at the National Research Council of Canada (NRC) for the Russian-English news translation task of the First Conference on Machine Translation (WMT 2016). Our submission is a phrase-based SMT system that tackles the morphological complexity of Russian through comprehensive use of lemmatization. The core of our lemmatization strategy is to use different views of Russian for different SMT components: word alignment and bilingual neural network language models use lemmas, while sparse features and reordering models use fully inflected forms. Some components, such as the phrase table, use both views of the source. Russian words that remain out-ofvocabulary (OOV) after lemmatization are transliterated into English using a statistical model trained on examples mined from the parallel training corpus. The NRC Russian-English MT system achieved the highest uncased BLEU and the lowest TER scores among the eight participants in WMT 2016.

Challenges in Evaluating Summaries of Short Stories

This paper presents experiments with the evaluation of automatically produced summaries of literary short stories. The summaries are tailored to a particular purpose of helping a reader decide whether she wants to read the story. The evaluation procedure includes extrinsic and intrinsic measures, as well as subjective and factual judgments about the summaries pronounced by human subjects. The experiments confirm the experience of summarizing more conventional genres: sentence overlap between human- and machine-made summaries is not a complete picture of the quality of a summary. In fact, in our case, sentence overlap does not correlate well with human judgment. We explain the evaluation procedures and discuss several challenges of evaluating summaries of works of fiction.