Yumiko Fukuda

Multiphasic Analysis of the Basic Nature of Speechreading

This report describes a series of analyses on the basic nature of speechreading, which cover such topics as perception of speech through substitutive sensation, coordination of the vowel chart and the mouth shape, mouth shape synthesis by contraction of the lip muscles, auditorily and visually transmitted speech information, a recording format of the movement of the lips, symbolic descriptive system of mouth shape, combined use of auditory and visual sensations, and a system of manual signs as an aid for speechreading.

Description of mouth shapes used in traditional Japanese sign language by comparison to those of speech utterance

Mouth shapes used in traditional Japanese sign language were described systematically through comparison to those in uttered speech. A framework of mouth shape descriptions that had been proposed by the present authors included (1) projection of mouth shapes of Japanese vowels and semi‐vowels to the lip articulation plane of the three‐dimensional IPA vowel chart and (2) grouping of discriminating consonants in lip‐reading on the IPA articulation matrix. It was transferred to a coordinate consisting of contractions of muscles involved in mouth movement. In traditional Japanese sign language, the mouth shape, when added as a part of facial expression, modifies meaning of the word conveyed by a hand gesture. Those mouth shapes were extracted from conversations among native signers and were symbolized by their respective degrees of closing, rounding, and pulling corners, and by categories borrowed from Japanese vowels and some consonants. Using the extended framework, those mouth shapes that were elected for visual discrimination ranged more widely than those elected to discriminate phonemes aurally. Video pictures were retrieved according to their symbols from an electronic dictionary of 250 basic sign words and 1500 exemplary sentences of their meanings, as edited by the Japanese sign language research group, NRCD.Mouth shapes used in traditional Japanese sign language were described systematically through comparison to those in uttered speech. A framework of mouth shape descriptions that had been proposed by the present authors included (1) projection of mouth shapes of Japanese vowels and semi‐vowels to the lip articulation plane of the three‐dimensional IPA vowel chart and (2) grouping of discriminating consonants in lip‐reading on the IPA articulation matrix. It was transferred to a coordinate consisting of contractions of muscles involved in mouth movement. In traditional Japanese sign language, the mouth shape, when added as a part of facial expression, modifies meaning of the word conveyed by a hand gesture. Those mouth shapes were extracted from conversations among native signers and were symbolized by their respective degrees of closing, rounding, and pulling corners, and by categories borrowed from Japanese vowels and some consonants. Using the extended framework, those mouth shapes that were elected for ...

CAI system for testing and training discrimination ability of change in voice pitch through the cochlear implant

In the cochlear implant, change in voice pitch is transmitted through the periodicity of change in density of neural pulse trains excited by the electrical signal. As the transmission quality of the voice pitch depends on the setting of the processor of each device, it is necessary to measure precisely the discrimination ability of change in voice pitch for each user, and to make maximum use of it. For this purpose, a CAI (Computer Assisted Instruction) system for testing and training the discrimination was developed.

Use of language media for social activities of hearing‐impaired persons in Japan

This selective use of language media by the hearing impaired in contemporary Japanese society was investigated. The data used in this study were collected from a survey conducted by mailing questionnaires to 1700 profoundly deaf males and females, aged from 20–70, in 1991. Responses to the items regarding the significance of the language media such as oral and manual methods, and conversation by writing were analyzed, with regard to the difference between the responses of both genders. The following results were obtained: (1) According to the category of people with whom they communicated, the rate of use and the rate of effectiveness of each medium were characteristically different; (2) according to the different occasions of the use of language in daily life, use of the language media and their efficiency differed significantly; (3) assistance from sign interpreters and accompanying persons was useful, depending on the occasion; (4) younger subjects showed higher ability in the use of language media tha...

Test materials for evaluating speech perception ability of cochlear implant patients

A system for assembling the test materials that are used in evaluating speech perception ability by combined use of cochlear implant and lip‐reading has been developed. In order to construct balanced lists, a database of Japanese basic vocabulary furnished with specifications regarding their auditory, visual, temporal, and linguistic components is utilized in this system. Factors whose distribution are balanced between each other list are the kind of vowel and the manner/place of articulation of the consonant, special Japanese syllables (elongated vowel, choked sound, and syllabic nasal), phones having the same mouth shape, number of syllables in a word and phrases in a sentence, class of familiarity, and grammatical data such as part of speech. All these data and specification of category of meaning of words in the database are used in searching suitable words for the lists, as well as in analyzing characteristics of perception ability of the patients. Problems in requirements for the evaluation methods,...

An analysis of the characteristics of speech perception ability by the combined use of cochlear implant and lipreading

In this study, test materials for evaluating speech perception ability by a combined use of cochlear implants and lipreading were first selected, taking into account various acoustical parameters such as fundamental frequency, intensity, and formant frequencies that can be transmitted through a speech signal processor in a cochlear implant, as well as supplemental speech information that can be obtained through mouth shape in lipreading. The test materials included the short versus long vowels, unvoiced versus voiced consonants. Japanese word accent and sentence intonation, and the 5 Japanese vowels, 50 monosyllables, 50 words, and 29 sentences that consisted of 100 words. Then, both the speech signal and the mouth shape for utterances of the test materials by a female speaker were recorded on videotapes and were presented to the subjects who had cochlear implants in Tokyo. The results showed that the test materials were useful for a detailed analysis of the characteristics of speech perception ability th...

Computer simulation of the fitting of hearing aids based on prediction of the discrimination ability of speech sounds

The earlier version of the computer program for simulating the process of fitting hearing aids [S. Hiki and Y. Fukuda, “Prediction of the ability of discrimination of speech sounds by hearing impaired children,” J. Acoust. Soc. Am. Suppl. 1 68, S5 (1980)] has been modified so that correction for individual speaker differences can be made. The latest version of the program consists of two parts. In the first part, by inputting various characteristics of the hearing loss of an impaired child and the candidate hearing aid, parameters of the hearing aid are controlled. The goal of the adjustment is to match the sound pressure versus frequency range, audible for each impaired child with the aided characteristics, to that of typical speech sounds. In the second part, the type of identifiable phones and probable phonetic confusions among the phones are estimated by analyzing the relationship between the aided characteristics and the acoustical properties of the speech sounds. The corrections for individual speak...

Prediction of the ability of discrimination of speech sounds by hearing impaired children

A computer program designed for predicting the acoustical limitation of discrimination of speech sounds by hearing impaired children is described. In this program, the kind of identifiable phones and the probable confusion among the phones are estimated based on the aided frequency characteristic of each individual hearing impaired child. The program is currently being used to find the most appropriate methods of prescribing hearing aids and evaluating use of the aided characteristics, especially in the courses of extending the highest limit of the frequency range or depressing the low frequency components and of utilizing directional microphones and telephone pickup coils mounted in the hearing aids.