Kotaro Fukui

Development of a New Human-like Talking Robot for Human Vocal Mimicry

This paper describes development of a new human-like talking robot WT-4 (Waseda Talker No. 4). WT-4 was developed to overcome the problems related to the lack of variation in the formant frequency of the sounds generated by the previous talking robot WT-3. These problems arose particularly on the first formant (F1) where frequencies of less than 500 [Hz] could not be reached. And the differences among WT-3’s vowels were not clear; which may have been caused by the sound source of the vocal cords and the lips. Therefore, the connection between the vocal cords and the vocal tract was improved, as well as the lips mechanism. As the result of these improvements, WT-4’s sounds were quite similar to a human’s, and WT-4 was able to reproduce suitable sounds using auditory feedback.

Three dimensional tongue with liquid sealing mechanism for improving resonance on an anthropomorphic talking robot

We have developed a new, three dimensional vocal tract mechanical model for an anthropomorphic talking robot WT-7R (Waseda Talker No. 7 Refined), to improve the resonance of the vocal tract. The Waseda Talker robot series aims to reproduce the human speech mechanism with three-dimensional accuracy. The tongue of the previous model, WT-7 (Waseda Talker No. 7), was made of rigid links and covered with a thermoplastic rubber (Septon). This mechanism could deform the tongue shape and work as a part of the vocal tract, however, the cover thickness was not sufficient enough to prevent sound leakage, and did not have sufficient resonance. As a result, the produced sounds were unclear. To resolve this problem, the inner area of the tongue was filled with liquid. We experimented to select the filling liquid which is minimizes damage to the Septon cover and provides adequate resonance characteristics. The ethylene glycol was selected because it does little damage to the Septon and is relatively non-flammable. An oil seal and liquid gasket prevent leakage into the robot. WT-7 also has problems with its tongue link deformation range and open lip control range—these problems were also addressed. The improvements made the vowel production of WT-7R clearer than that of the previous robot, and the bandwidth of the formant peak in spectral analysis became sharper.

Development of a human-like sensory feedback mechanism for an anthropomorphic talking robot

We developed a sensor feedback mechanism for an anthropomorphic talking robot WT-5 (Waseda Talker No. 5). In human speech, sensory feedback is more important when we producing obstacle consonant sounds, such as /t/ and /d/, compared to the auditory feedback mechanism. We reproduce this mechanism by placing tactile sensors and a pressure sensor on the palate of a talking robot and reducing the error between the pressure of the human voice and the robot consonant production. In addition, we developed more efficient optimization methods than those of WT-4, using speech recognition and the pre-optimized memory of vowels. Using these mechanisms, we realized continuous mimic speaking that includes consonant sounds

New Anthropomorphic Talking Robot having a Three-dimensional Articulation Mechanism and Improved Pitch Range

We have developed a new three-dimensional talking robot Waseda Talker No. 6 (WT-6), which generates speech sounds by mechanically simulating articulatory motions as well as aero-acoustic phenomena in the vocal tract. WT-6 has 17-DOF vocal mechanism. It has three-dimensional lips, tongue, jaw and velum which form the 3D vocal tract structure. It also has an independent jaw opening/closing mechanism, which controls the relative tongue position in the vocal tract as well as the oral opening. The previous robot in the series had a 2D tongue and was not able to realize precise closure to produce human-like consonants such as /s/ or /r/. The new tongue, which can be controlled to form 3D shapes, is able to produce more realistic vocal tract shapes. The vocal cord model was also improved by adding a new pitch control mechanism that pushes from the side of the vocal cords. The pitch range is broader than that of the previous robot; it is sufficiently broad so as to be able to reproduce normal human speech. Preliminary experimental results showed improved synthesized speech quality for the vowels /a/, /u/ and /o/.

Control methods based on neural network forward and inverse models for a biomechanical structured vocal cord model on an anthropomorphic talking robot

We have developed a vocal control method, based on forward and inverse models, to allow the anthropomorphic talking robot Waseda Talker No. 7 (WT-7) to produce various kinds of voices. The control parameters of the vocal cords on WT-7 are pressure, vocal cord tension and glottal opening, and the acoustic parameters are sound pressure, sound pitch and spectrum slope. The relationships among these parameters are complicated and difficult to model using conventional methods. Here we present a neural network (NN) control method. The learning process consists of creation of the NN forward model by back propagation methods and optimization of the inverse model using the forward model. In addition, a real-time auditory feed-back mechanism is used to reduce the error between the target and the generated acoustic parameters. Using this method, the control parameters can be adjusted to follow the target voice well.

New Anthropomorphic Talking Robot having Sensory Feedback Mechanism and Vocal Cords based on Human Biomechanical Structure

We developed an anthropomorphic talking robot WT-5 (Waseda Talker No. 5) having a sensor feedback mechanism and novel vocal cords based on human biological structures. The sensory feedback is more important when we producing obstacle consonant sounds, compared to the auditory feedback mechanism. We reproduce this mechanism by placing tactile sensors and a pressure sensor on the palate of a talking robot and reducing the error between the pressure of the human voice and the robot consonant production. The vocal cord model was constructed with a similar structure to the biological structure of the human vocal cords and vibrated like those of a human. In addition, using these mechanisms, WT-5 could produce voices closer to those of a human. And also realized continuous mimic speaking that includes consonant sounds

Anthropomorphic Talking Robot Based on Human Biomechanical Structure

We developed an anthropomorphic talking robot, Waseda Talker No. 6 (WT-6), which generates speech sounds by mechanically simulating articulatory motions and aero-acoustic phenomena. WT-6 possesses 17 degrees of freedom (DOF): a 5-DOF tongue, 1-DOF jaws, 4-DOF lips, a nasal cavity, and a 1-DOF soft palate as articulators; and 5-DOF vocal cords and 1-DOF lungs as vocal organs. The vocal cords, tongue, and lips are made from the thermoplastic rubber Septon, whose elasticity is similar to that of human tissue. WT-6 has three-dimensional (3D) lips, tongue, jaw, and velum, which form the vocal tract structure. It also has an independent jaw opening/closing mechanism. The previous robot in the series had a two-dimensional tongue and could not produce human-like tongue shape. The new tongue can form 3D shapes, and thus, is able to produce more realistic vocal tract shapes. The vocal cord model consists of two folds, and is constructed with a structure similar to the biomechanical structure of human vocal cords. These vocal cords can vibrate in complex phases, similar to those of a human. With these mechanisms, the robot can reproduce human speech in a more biomechanical manner, and thus, can produce a voice closer to that of a human.

Development of a new vocal cords based on human biological structures for talking robot

We developed a new talking robot, WT-5 (Waseda Talker No. 5), having novel vocal cords, based on human biological structures. The vocal cords were made from the thermoplastic rubber “Septon”, available from Kuraray Co. Ltd. Septon has a similar elasticity to human tissue. The vocal cord model to have a structure similar to the biological structure of the human vocal cords was made. The vocal cords were vibrated like those of a human. This made clean the robots vowels. With these new mechanisms, the robot could reproduce the human speech in a more biological view and could produce voices nearer to those of a human.

Mechanical vocal cord model mimicking human biological structure

We present a mechanical vocal cord model aiming for a talking robot, WT‐5 (Waseda Talker No. 5). Unlike a musical reed which has been used in conventional mechanical speech synthesizer, the vocal cord model is formed to mimic the human’s vocal cord in the shape and the biological structure. It is made of a thermoplastic rubber, Septonh (Kuraray Co. Ltd.) of which the elasticity like a human’s, and has 3‐DOF mechanisms which is similar to the human structure. 1‐DOF link mechanism could change the pitch by stretching the length of the vocal cords. The 2‐DOF arm mechanism is used to mimic the abduction and adduction of a human arytenoid cartilage. The vocal cord model was excited by air flow exhausted from a mechanical lung model. The vibration pattern was observed by a high‐speed camera, and the glottal volume velocity and the sound pressure were recorded by a mask‐type wire screen pneumotachograph and a microphone. It was shown that the lower and upper edges of the vocal cords could vibrate in a different ...

Production of various vocal cord vibrations using a mechanical model for an anthropomorphic talking robot

We developed a three-dimensional mechanical vocal cord model for Waseda Talker No. 7 (WT-7), an anthropomorphic talking robot, for generating speech sounds with various voice qualities. The vocal cord model is a cover model that has two thin folds made of thermoplastic material. The model self-oscillates by airflow exhausted from the lung model and generates the glottal sound source, which is fed into the vocal tract for generating the speech sound. Using the vocal cord model, breathy and creaky voices, as well as the modal (normal) voice, were produced in a manner similar to the human laryngeal control. The breathy voice is characterized by a noisy component mixed with the periodic glottal sound source and the creaky voice is characterized by an extremely low-pitch vibration. The breathy voice was produced by adjusting the glottal opening and generating the turbulence noise by the airflow just above the glottis. The creaky voice was produced by adjusting the vocal cord tension, the sub-glottal pressure and the vibration mass so as to generate a double-pitch vibration with a long pitch interval. The vocal cord model used to produce these voice qualities was evaluated in terms of the vibration pattern as measured by a high-speed camera, the glottal airflow and the acoustic characteristics of the glottal sound source, as compared to the data for a human.