Simon Petitrenaud

Combination, cooperation and selection of classifiers: a state of the art

When several classifiers are brought to contribute to the same task of recognition, various strategies of decisions, implying these classifiers in different ways, are possible. A first strategy consists in deciding using different opinions: it corresponds to the combination of classifiers. A second strategy consists in using one or more opinions for better guiding other classifiers in their training stages, and/or to improve the decision-making of other classifiers in the classification stage: it corresponds to the cooperation of classifiers. The third and last strategy consists in giving more importance to one or more classifiers according to various criteria or situations: it corresponds to the selection of classifiers. The temporal aspect of Pattern Recognition (PR), i.e. the possible evolution of the classes to be recognized, can be treated by the strategy of selection.

Automatic named identification of speakers using diarization and ASR systems

In this paper, we consider the extraction of speaker identity from audio records of broadcast news without a priori acoustic information about speakers. Using an automatic speech recognition system and an automatic speaker diarization system, we present improvements for a method which allows to extract speaker identities from automatic transcripts and to assign them to speech segments. Experiments are carried out on French broadcast news records from the ESTER 1 evaluation campaign. Experimental results using outputs of automatic speech recognition and automatic diarization are presented.

HAMEX - A Handwritten and Audio Dataset of Mathematical Expressions

In this paper, we present HAMEX, a new public dataset that contains mathematical expressions available in their on-line handwritten form and in their audio spoken form. We have designed this dataset so that, given a mathematical expression, its handwritten signal and its audio signal can be used jointly to design multimodal recognition systems. Here, we describe the different steps that allowed us to acquire this dataset, from the creation of the mathematical expression corpora (including expressions from Wikipedia pages) to the segmentation and the transcription of the collected data, via the data collection process itself. Currently, the dataset contains 4 350 on-line handwritten mathematical expressions written by 58 writers, and the corresponding audio expressions (in French) spoken by 58 speakers. The ground truth is also provided both for the handwritten expressions (as INKML files with the digital ink, the symbol segmentation, and the MATHML structure) and for the audio expressions (as XML files with the transcriptions of the spoken expressions).

Handling Different Forms of Uncertainty in Regression Analysis: A Fuzzy Belief Structure Approach

We propose a new approach to functional regression based on fuzzy evidence theory. This method uses a training set for computing a fuzzy belief structure which quantifies different types of uncertainties, such as nonspecificity, conflict, or low density of input data. The method can cope with a very large class of training data, such as numbers, intervals, fuzzy numbers, and, more generally, fuzzy belief structures. In order to limit calculations and improve output readability, we propose a belief structure simplification method, based on similarity between fuzzy sets and significance of these sets. The proposed model can provide predictions in several different forms, such as numerical, probabilistic, fuzzy or as a fuzzy belief structure. To validate the model, we propose two simulations and compare the results with classical or fuzzy regression methods.

What types of word alignment improve statistical machine translation

In most statistical machine translation (SMT) systems, bilingual segments are extracted via word alignment. However, there is a need for systematic study as to what alignment characteristics can benefit MT under specific experimental settings such as the type of MT system, the language pair or the type or size of the corpus. In this paper we perform, in each of these experimental settings, a statistical analysis of the data and study the sample correlation coefficients between a number of alignment or phrase table characteristics and variables such as the phrase table size, the number of untranslated words or the BLEU score. We report results for two different SMT systems (a phrase-based and an n-gram-based system) on Chinese-to-English FBIS and BTEC data, and Spanish-to-English European Parliament data. We find that the alignment characteristics which help in translation greatly depend on the MT system and on the corpus size. We give alignment hints to improve BLEU score, depending on the SMT system used and the type of corpus. For example, for phrase-based SMT, dense alignments are required with larger corpora, especially on the target side, while with smaller corpora, more precise, sparser alignments are better, especially on the source side. Avoiding some long-distance crossing links may also improve BLEU score with small corpora. We take these conclusions into account to modify two types of alignment systems, and get 1 to 1.6 % relative improvements in BLEU score on two held-out corpora, although the improved system is different in each corpus.

Handwritten and Audio Information Fusion for Mathematical Symbol Recognition

Considerable efforts are being done within the scientific community to make as easier as possible the way that the human being converses with its machine. Handwriting and speech are two common ways used to achieve this goal and are probably among those which attracted much interest. In mathematical content recognition tasks, these two modalities are used with a certain success. This paper presents an architecture based on a speech handwriting data fusion for isolated mathematical symbol recognition. Different fusion methods are explored. The results are very encouraging since recognition rates are increased comparatively to mono modality approaches.

Combining transcription-based and acoustic-based speaker identifications for broadcast news

In this paper, we consider the issue of speaker identification within audio records of broadcast news. The speaker identity information is extracted from both transcript-based and acoustic-based speaker identification systems. This information is combined in the belief functions framework, which makes coherent the knowledge representation of the problem. The Kuhn-Munkres algorithm is used to optimize the assignment problem of speaker identities and speaker clusters. Experiments carried out on French broadcast news from the French evaluation campaign ESTER show the efficiency of the proposed combination method.

Identification of Speakers by Name Using Belief Functions

In this paper, we consider the extraction of speaker identity (first name and last name) from audio records of broadcast news. Using an automatic speech recognition system, we present improvements for a method which allows to extract speaker identities from automatic transcripts and to assign them to speaker turns. The detected full names are chosen as potential candidates for these assignments. All this information, which is often contradictory, is described and combined in the Belief Functions formalism, which makes the knowledge representation of the problem coherent. The Belief Function theory has proven to be very suitable and adapted for the management of uncertainties concerning the speaker identity. Experiments are carried out on French broadcast news records from a French evaluation campaign of automatic speech recognition.

Text Alignment from Bimodal Mathematical Expression Sources

In this paper we propose a new approach to merge mathematical expression recognition results coming from handwriting and speech modalities. Using a bimodal description of mathematical expressions allows taking advantage of the complementarities between both signals, and can disambiguate situations were a single modality would not be clear enough. To combine the signals coming from both modalities, we propose to represent them in the same space as a textual description. First, from the handwriting signal, we generate the Nbest mathematical expressions, each of them is next translated as different possible strings. From the audio signal, an automatic speech recognition system provides a transcript, which is also available as a string. A string comparison algorithm is achieved to select the best mathematical expressions. This bimodal system is evaluated on real bimodal data from the HAMEX dataset and the results are compared to a single modality (handwriting) based system.

Multimodal mathematical expressions recognition: case of speech and handwriting

In this work, we propose to combine two modalities, handwriting and speech, to build a mathematical expression recognition system. Based on two sub-systems which process each modality, we explore various fusion methods to resolve ambiguities which naturally occur independently. The results that are reported on the HAMEX bimodal database show an improvement with respect to a mono-modal based system.The work reported in this paper concerns the problem of mathematical expressions recognition. This task is known to be a very hard one. We propose to alleviate the difficulties by taking into account two complementary modalities. The modalities referred to are handwriting and audio ones. To combine the signals coming from both modalities, various fusion methods are explored. Performances evaluated on the HAMEX dataset show a significant improvement compared to a single modality (handwriting) based system.