Kenkichi Ishizuka

Combination analysis of motion and melody in phrase animation

Impressions of phrase motion and those of a melody are expressed by adjectives and their combinations for the phrase animation generation are analyzed in this paper. Factors in a sound impression space and those in a phrase motion impression space are extracted by the factor analysis using data obtained by the pre-experiments. Melody and phrase motion database are constructed using data obtained by the pre-experiments. The melody candidate and the phrase motion candidate are chosen according to the impression value on the sound impression space and that on the motion impression space, which are obtained using the concept of co-occurrence of adjectives. Subject experiments are performed to analyze the combination of phrase motion and a melody for the phrase animation generation. The following three combinations are considered: (1) Impressions of phrase motion and those of a melody are expressed by same adjectives. (2) Impressions of phrase motion and those of a melody are expressed by opposite adjectives. (3) Impressions of phrase motion and those of melody are expressed by adjectives giving similar impressions. Experimental results show that combination (1) and combination (3) are effective.

Generation of variations on theme music based on impressions of story scenes considering human's feeling of music and stories

This paper describes a system which generates variations on theme music fitting to story scenes represented by texts and/or pictures. Inputs to the present system are original theme music and numerical information on given story scenes. The present system varies melodies, tempos, tones, tonalities, and accompaniments of given theme music based on impressions of story scenes. Genetic algorithms (GAs) using modular neural network (MNN) models as fitness functions are applied to music generation in order to reflect users feeling of music and stories. The present system adjusts MNN models for each user on line. This paper also describes the evaluation experiments to confirm whether the generated variations on theme music reflect impressions of story scenes appropriately or not.

Automatic scoring method considering quality and content of speech for scat Japanese speaking test

We are now developing a Japanese speaking test called SCAT, which is part of J-CAT (Japanese Computerized Adaptive Test), a free online proficiency test for Japanese language learners. In this paper, we focus on the sentence-reading-aloud task and the sentence generation task in SCAT, and propose an automatic scoring method for estimating the overall score of answer speech, which is holistically determined by language teachers according to a rating standard. In that process, teachers carefully consider different factors but do not rate the scores of them. We therefore analyze how each factor contributes to the overall score. The factors are divided into two categories: the quality of speech and the content of speech. The former includes pronunciation and intonation, and the latter representation and vocabulary. We then propose an automatic scoring method based on the analysis. Experimental results confirm that the proposed method gives relatively accurate estimates of the overall score.

Automatic scoring method for open answer task in the SJ-CAT speaking test considering utterance difficulty level

In this paper, we propose an automatic scoring method for the open answer task of the Japanese speaking test SJ-CAT. The proposed method first extracts a set of features from an input answer utterance and then estimates a vocabulary richness score by human raters, which ranges from 0 to 4, by employing SVR (support vector regression). We devised a novel set of features, namely text statistics weighted by word reliability, to assess the abundance of vocabulary and expression, and degree of word relevance based on the hierarchical distance in a thesaurus to evaluate the suitability of vocabulary. We confirmed experimentally that the proposed method provides good estimates of the human richness score, with a correlation coefficient of 0.92 and an RMSE (root mean square error) of 0.56. We also showed that the proposed method is relatively robust to differences among examinees and among questions used for training and testing.

Analysis of theatrical effects in multimedia picture story show

Impressions of combinations of lighting patterns, lyrics and music are analyzed for designing the system which generates the theatrical effects using the three modal media based on adjectives representing producers impressions of story scenes. Impression spaces of three modal media are constructed by factor analysis using data obtained by pre-experiments. The three modal media are generated according to the coordinates in the impression spaces of three modal media, where the coordinates are estimated using the concept of co-occurrence of inputted adjectives. In this paper, subject experiments are performed to analyze the interaction of combinations of the three modal media. The following four combinations are considered: (1) combinations based on same adjectives, (2) combinations based on counter adjectives, (3) combinations based on similar adjectives, (4) combinations based on dissimilar adjectives. Experimental results show that combinations (1),(3), and (4) are effective as theatrical effects of story scenes.