Deirdre Hogan

Parser-based retraining for domain adaptation of probabilistic generators

While the effect of domain variation on Penn-treebank-trained probabilistic parsers has been investigated in previous work, we study its effect on a Penn-Treebank-trained probabilistic generator. We show that applying the generator to data from the British National Corpus results in a performance drop (from a BLEU score of 0.66 on the standard WSJ test set to a BLEU score of 0.54 on our BNC test set). We develop a generator retraining method where the domain-specific training data is automatically produced using state-of-the-art parser output. The retraining method recovers a substantial portion of the performance drop, resulting in a generator which achieves a BLEU score of 0.61 on our BNC test data.

SMS Normalisation, Retrieval and Out-of-Domain Detection Approaches for SMS-Based FAQ Retrieval

This paper gives an overview of DCU’s participation in the SMS-based FAQ Retrieval task at FIRE 2011. DCU submitted three runs for monolingual English experiments. The approach consisted of first transforming the noisy SMS queries into a normalised, corrected form. The normalised queries were then used to retrieve a ranked list of FAQs by combining the results from three different retrieval methods. Finally, using information from the retrieval results, out-of-domain (OOD) queries were identified and tagged. The results of our best run on the final test set were the highest of all 13 participating teams. Our FIRE submission retrieved 70.2% in-domain query answers correctly and 85.6% identified out-of-domain queries correctly.