Yoichi Takebayashi

Speech dialogue system for facilitating improved human-computer interaction

A speech dialogue system capable of realizing natural and smooth dialogue between the system and a human user, and easy maneuverability of the system. In the system, a semantic content of input speech from a user is understood and a semantic content determination of a response output is made according to the understood semantic content of the input speech. Then, a speech response and a visual response according to the determined response output are generated and outputted to the user. The dialogue between the system and the user is managed by controlling transitions between user states during which the input speech is to be entered and system states during which the system response is to be outputted. The understanding of a semantic content of input speech from a user is made by detecting keywords in the input speech, with the keywords to be detected in the input speech limited in advance, according to a state of a dialogue between the user and the system.

Speech recognition apparatus using syntactic and semantic analysis

A speech recognition apparatus comprises a speech input unit for receiving an input speech signal, analyzing it, and outputting a speech feature parameter series, a speech recognition unit for extracting a speech feature vector from the parameter series, and matching it with a plurality of predetermined words to output a series of word candidates used as keywords, a syntactic analysis unit for analyzing the series of the word candidates as the keywords according to syntactic limitation, and generating a sentence candidate.

Speech recognition system having word-based and phoneme-based recognition means

A speech recognition system includes a parameter extracting section for extracting a speech parameter of input speech, a first recognizing section for performing recognition processing by word-based matching, and a second recognizing section for performing word recognition by matching in units of word constituent elements. The first word recognizing section segments the speech parameter in units of words to extract a word speech pattern and performs word recognition by matching the word speech pattern with a predetermined word reference pattern. The second word recognizing section performs recognition in units of word constituent elements by using the extracted speech parameter and performs word recognition on the basis of candidates of an obtained word constituent element series. The speech recognition system further includes a recognition result output section for obtaining a recognition result on the basis of the word recognition results obtained by the first and second recognizing sections and outputting the obtained recognition result. The speech recognition system further includes a word reference pattern learning section for performing learning of a word reference pattern on the basis of the recognition result obtained by the recognizing result output section and the word speech pattern.

Learning system of dictionary for speech recognition

The learning method of reference pattern vectors for speech recognition in accordance with the present invention, a plurality of speech feature vectors are generated from the time series of speech feature parameter for the input speech pattern, by taking into account knowledge concerning the variation tendencies of the speech patterns, and the learning (preparation) of reference pattern vectors for speech recognition is carried out by the use of these speech feature vectors thus generated. Therefore, it becomes possible to prepare highly reliable reference pattern vectors in an easy manner from a small number of speech patterns, which makes it possible to achieve an improvement in the speech recognition factor. In particular, it becomes possible to plan an easy improvement of the reference pattern vectors by an effective use of a relatively small number of input speech patterns.

A real-time task-oriented speech understanding system using keyword-spotting

The authors describe a real-time task-oriented spontaneous speech understanding system which extracts semantic content from the spotted keyword lattice using a newly developed LR parser. The parser is driven whenever a keyword candidate is spotted. The keyword candidates are fed into the parser to construct a semantic structure using a grammar. This efficient parsing enables real-time initial-state-free speech understanding. The parser comprises the following functional components: initial-state processing to check if a keyword candidate can be an initial keyword and to create a parsing stack, keyword connection processing to check if a current keyword can connect with a subsentence candidate, and sentence acceptance processing to check if subsentence candidates can be accepted as a sentence candidate. Experimental results for a fast-food task consisting of 30 keywords are presented to show the effectiveness of the system.<<ETX>>

Keyword-spotting in noisy continuous speech using word pattern vector subabstraction and noise immunity learning

Noise immunity learning, previously proposed by the authors (1991) for isolated word recognition in noisy environments, is extended to keyword spotting in noisy continuous speech. The powerful features of the noise immunity keyword-spotting method are keyword spotting based on the multiple similarity (MS) method for reliable keyword detection, noise immunity learning for greater robustness in recognition of spontaneous or noisy speech, and word pattern vector subabstraction to represent noisy keyword patterns from different viewpoints. Integrating the spotting results obtained by different kinds of subabstracted word pattern vectors significantly improved the performance of the keyword spotting. A system to spot 30 keywords currently runs in real-time on a workstation with two accelerators. The spotted keywords are fed into a keyword sequence LR parser for spontaneous speech understanding.<<ETX>>

Spontaneous speech dialogue system TOSBURG II and its evaluation

Abstract We have developed a spontaneous speech dialogue system TOSBURG II, employing keyword-based spontaneous speech understanding and multimodal response generation, with adaptive speech response cancellation. Since in multimodal interaction, the user understands the systems response by a visual output before its speech response is completed, the user often interrupts the systems speech response. Therefore, our adaptive speech response cancellation serves to facilitate natural human-computer interaction by allowing the users interruption. We have also developed an evaluation environment for dialogue data collection and the performance of TOSBURG II. Unlike conventional data collection systems, TOSBURG II collects in this environment not only speech data and the final results of speech understanding but also its intermediate results as dialogue data, to use them for the evaluation and improvement of the system. The results of our dialogue experiments using TOSBURG II prove the effectiveness of adaptive speech response cancellation for natural interaction, confirming that the dialogue data and the evaluation environment will contribute to a further development of spontaneous speech dialogue systems.

Method and apparatus for speech recognition using both low-order and high-order parameter analyzation

A speech recognition apparatus includes a a low-order parameter analyzation section for deriving low-order parameter time series of an input speech, a start and end point detection section for detecting the start and end points of the input speech by use of the parameter time series derived by the low-order parameter analyzation section, a high-order parameter analyzation section for deriving high order parameters at preset sampling points from the input speech of a range corresponding to the start and end points detected by the start and end point detection section, a pattern matching section for matching feature parameters corresponding to the parameters derived by the high-order parameter analyzation section with standard parameters previously registered and deriving similarities between the matched parameters, and a recognition result outputting section for outputting the recognition result for the input speech according to the similarities derived by the pattern matching section.

A hybrid wordspotting method for spontaneous speech understanding using word-based pattern matching and phoneme-based HMM

We have proposed a new wordspotting method, combining word-based pattern matching and phoneme-based hidden Markov model (HMM). Word-based pattern matching based on the time-frequency representation of a whole word pattern is robust against pattern variations and background noise, while the phoneme-based HMM, which represents phonemic features within a word pattern, is flexible for expanding the vocabulary. Because of the difference in scope, these two have their own characteristics in terms of robustness and accuracy. To take advantage of the features of these two, we have integrated these different types of wordspotting results under a unified criterion. A syntactic and semantic parser is also utilized to prune the wordspotting results for spontaneous speech understanding. Experimental results indicate the effectiveness of the proposed method.

Noisy spontaneous speech understanding using noise immunity keyword spotting with adaptive speech response cancellation

The system described is an extension of TOSBURG (taste-oriented speech dialogue system based on speech understanding and response generation), TOSBURG II. To facilitate robust human-computer interaction, a new component has been introduced: adaptive cancellation of synthetic speech response. In addition, noise immunity learning has been extended to dialogue speech contaminated by an uncancelled component of a synthetic speech response. The cancellation allows the user to interrupt speech responses of the system for more efficient human-computer interaction. A user-initiated dialogue manager and a multimodal response generator have been integrated to construct a speaker-independent dialogue system for a fast-food ordering task. Experimental results have shown the effectiveness of adaptive speech response cancellation and the extension of noise immunity learning.<<ETX>>