Yoshua Bengio

Phonetically motivated acoustic parameters for continuous speech recognition using neural networks

Abstract In the framework of an ANN/HMM hybrid system for phone recognition three specialized ANNs were designed and evaluated. One of these ANNs detects the manner of articulation. The other two ANNs describe the speech signal in terms of place of articulation. One of these is used for plosive and nasal classification, and the other one is used for fricative classification. The design of these networks was inspired by acoustic-phonetic knowledge. Input parameters, ANN topology and desired output representation have been optimized for the specific task of the network. Experiments are reported for the TIMIT database. Frame classification errors of 17.7% with the manner ANN (5 broad classes), 25.4% with the plosive and nasal ANN (10 phones), and 25.2% with the fricative ANN (11 phones) were obtained on a set of 616 sentences from 77 new speakers. Experiments for a prototype ANN/HMM hybrid system are also reported. We developed an algorithm for the global optimization of this hybrid system. The network for the manner of articulation and one network for the place of articulation were merged to a single ANN which outputs were modeled by an HMM. With this globally optimized hybrid system we achieved a recognition accuracy of 86% on an 8 class recognition problem (7 plosives and one class corresponding to all other phonemes).

The First Conversational Intelligence Challenge

The first Conversational Intelligence Challenge was conducted over 2017 with finals at NIPS conference. The challenge IS aimed at evaluating the state of the art in non-goal-driven dialogue systems (chatbots) and collecting a large dataset of human-to-machine and human-to-human conversations manually labelled for quality. We established a task for formal human evaluation of chatbots that allows to test capabilities of chatbot in topic-oriented dialogue. Instead of traditional chit-chat, participating systems and humans were given a task to discuss a short text. Ten dialogue systems participated in the competition. The majority of them combined multiple conversational models such as question answering and chit-chat systems to make conversations more natural. The evaluation of chatbots was performed by human assessors. Almost 1,000 volunteers were attracted and over 4,000 dialogues were collected during the competition. Final score of the dialogue quality for the best bot was 2.7 compared to 3.8 for human. This demonstrates that current technology allows supporting dialogue on a given topic but with quality significantly lower than that of human. To close this gap we plan to continue the experiments by organising the next conversational intelligence competition. This future work will benefit from the data we collected and dialogue systems that we made available after the competition presented in the paper.