João Paulo Neto

AUDIMUS.MEDIA: a broadcast news speech recognition system for the european portuguese language

Many applications such as media monitoring are experiencing a large expansion as a consequence of the different emerging media sources and can benefit dramatically by using automatic transcription of audio data. In this paper, we describe the development of a speech recognition engine, AUDIMUS.MEDIA used in the Broadcast News domain. Additionally we describe recent improvements that permitted a relative recognition error decrease of more than 20% and a 4x speed-up.

Non-speech audio event detection

Audio event detection is one of the tasks of the European project VIDIVIDEO. This paper focuses on the detection of non-speech events, and as such only searches for events in audio segments that have been previously classified as non-speech. Preliminary experiments with a small corpus of sound effects have shown the potential of this type of corpus for training purposes. This paper describes our experiments with SVM and HMM-based classifiers, using a 290-hour corpus of sound effects. Although we have only built detectors for 15 semantic concepts so far, the method seems easily portable to other concepts. The paper reports experiments with multiple features, different kernels and several analysis windows. Preliminary experiments on documentaries and films yielded promising results, despite the difficulties posed by the mixtures of audio events that characterize real sounds.

Broadcast news subtitling system in Portuguese

The subtitling of broadcast news programs are starting to become a very interesting application due to the technological advances in automatic speech recognition and associated technologies. However, to build this kind of systems, several advances are necessary both in terms of the technological components and on main blocks integration. In this paper, we are presenting the overall architecture of a subtitling system running daily at RTP (the Portuguese public broadcast company). The goal is to integrate our components in a system for the subtitling of RTP programs. The global system includes the subtitling of recorded and direct programs.

Audio segmentation, classification and clustering in a broadcast news task

The paper describes our work on the development of an audio segmentation, classification and clustering system applied to a broadcast news task for the European Portuguese language. We developed a new algorithm for audio segmentation that is both accurate and uses fewer computational resources than other approaches. Our speaker clustering module uses a modified BIC (Bayesian information criterion) algorithm which performs substantially better than the standard symmetric Kullback-Liebler, KL2, and is much faster than the full BIC. Finally, we developed a scheme for tagging certain speaker clusters (anchors) using trained cluster models. A series of tests were conducted showing the advantage of the new algorithms. This system is part of a prototype system that is daily processing the main news show of the national Portuguese broadcaster.

Dynamic language modeling for a daily broadcast news transcription system

When transcribing Broadcast News data in highly inflected languages, the vocabulary growth leads to high out-of-vocabulary rates. To address this problem, we propose a daily and unsupervised adaptation approach which dynamically adapts the active vocabulary and LM to the topic of the current news segment during a multi-pass speech recognition process. Based on texts daily available on the Web, a story-based vocabulary is selected using a morpho-syntatic technique. Using an Information Retrieval engine, relevant documents are extracted from a large corpus to generate a story-based LM. Experiments were carried out for a European Portuguese BN transcription system. Preliminary results yield a relative reduction of 65.2% in OOV and 6.6% in WER.

A Prototype System for Selective Dissemination of Broadcast News in European Portuguese

This paper describes ongoing work on selective dissemination of broadcast news. Our pipeline system includes several modules: audio preprocessing, speech recognition, and topic segmentation and indexation. The main goal of this work is to study the impact of earlier errors in the last modules. The impact of audio preprocessing errors is quite small on the speech recognition module, but quite significant in terms of topic segmentation. On the other hand, the impact of speech recognition errors on the topic segmentation and indexation modules is almost negligible. The diagnostic of the errors in these modules is a very important step for the improvement of the prototype of a media watch system described in this paper.

Speaker-adaptation in a hybrid HMM-MLP recognizer

Presently the most important systems for large vocabulary, continuous speech recognition are speaker-independent. These systems deal with the inter-speaker variability through a large pool of speakers. However, this approach has several drawbacks due to its inability to cope with the individual speaker characteristics. The problem is more extreme for the cases of fast or non-native speakers. In this paper we present a technique for speaker-adaptation in the context of a hybrid HMM-MLP system for large vocabulary, speaker-independent, continuous speech recognition. This technique is implemented both in supervised and unsupervised modes. In the unsupervised case both static and incremental approaches are explored. The results show that speaker-adaptation within the hybrid HMM-MLP framework can substantially improve system performance. In the incremental unsupervised mode, the improvement is obtained without any extra demands on the speaker, i.e. without an enrolment phase.

Exploring events and distributed representations of text in multi-document summarization

We explore an event detection framework to improve multi-document summarizationWe use distributed representations of text to address different lexical realizationsSummarization is based on the hierarchical combination of single-document summariesWe performed an automatic evaluation and a human study of the generated summariesQuantitative and qualitative results show clear improvements over the state-of-the-art In this article, we explore an event detection framework to improve multi-document summarization. Our approach is based on a two-stage single-document method that extracts a collection of key phrases, which are then used in a centrality-as-relevance passage retrieval model. We explore how to adapt this single-document method for multi-document summarization methods that are able to use event information. The event detection method is based on Fuzzy Fingerprint, which is a supervised method trained on documents with annotated event tags. To cope with the possible usage of different terms to describe the same event, we explore distributed representations of text in the form of word embeddings, which contributed to improve the summarization results. The proposed summarization methods are based on the hierarchical combination of single-document summaries. The automatic evaluation and human study performed show that these methods improve upon current state-of-the-art multi-document summarization systems on two mainstream evaluation datasets, DUC 2007 and TAC 2009. We show a relative improvement in ROUGE-1 scores of 16% for TAC 2009 and of 17% for DUC 2007.

Self reinforcement for important passage retrieval

In general, centrality-based retrieval models treat all elements of the retrieval space equally, which may reduce their effectiveness. In the specific context of extractive summarization (or important passage retrieval), this means that these models do not take into account that information sources often contain lateral issues, which are hardly as important as the description of the main topic, or are composed by mixtures of topics. We present a new two-stage method that starts by extracting a collection of key phrases that will be used to help centrality-as-relevance retrieval model. We explore several approaches to the integration of the key phrases in the centrality model. The proposed method is evaluated using different datasets that vary in noise (noisy vs clean) and language (Portuguese vs English). Results show that the best variant achieves relative performance improvements of about 31% in clean data and 18% in noisy data.

Dynamic Vocabulary Adaptation for a daily and real-time Broadcast News Transcription System

The daily and real-time transcription of broadcast news (BN) is a challenging task both in acoustic and in language modeling. To achieve optimal performance, several problems have to be overcome. Particularly, when transcribing BN data in highly inflected languages, the vocabulary growth leads to high OOV word rates. To address this problem, we propose a daily vocabulary and LM adaptation framework which directly extracts new words based on contemporary written news available on the Internet and some linguistic knowledge about the words found on those news. Experiments have been carried out for a European Portuguese BN transcription system. Preliminary results computed on 7 shows, yields a relative reduction of 61% in OOV and 2.1% in WER.