Katsunori Kotani

Learner Feature Variation in Measuring the Listenability for Learners of English as a Foreign Language

Previous research on the ease of listening comprehension (henceforth, listenability) has measured listenability on the basis of sentence properties such as the length of words/sentences and speech rate. Recent research has included features of listeners, which are required for the measurement of listenability for English learners because their listening proficiencies vary greatly from the beginner to the advanced level. Given the importance of listening proficiency as a listener feature, this study developed listenability measurement methods based on the costs of compiling listener features: expensive features extracted from test scores and inexpensive features extracted from learners’ experiences. The experimental results showed that inexpensive features made substantial contributions to the measurement of middle-range listenability.

Validity of an Automatic Evaluation of Machine Translation Using a Word-Alignment-Based Classifier

Because human evaluation of machine translation is extensive but expensive, we often use automatic evaluation in developing a machine translation system. From viewpoint of evaluation cost, there are two types of evaluation methods: one uses (multiple) reference translation, e.g., METEOR, and the other classifies machine translation either into machine-like or human-like translation based on translation properties, i.e., a classification-based method. Previous studies showed that classification-based methods could perform evaluation properly. These studies constructed a classifier by learning linguistic properties of translation such as length of a sentence, syntactic complexity, and literal translation, and their classifiers marked high classification accuracy. These previous studies, however, have not examined whether their classification accuracy could present translation quality. Hence, we investigated whether classification accuracy depends on translation quality. The experiment results showed that our method could correctly distinguish the degrees of translation quality.

A machine learning-based evaluation method for machine translation

Constructing a classifier that distinguishes machine translations from human translations is a promising approach to automatic evaluation of machine-translated sentences Using this approach, we constructed a classifier using Support Vector Machines based on word-alignment distributions between source sentences and human or machine translations This paper investigates the validity of the classification-based method by comparing it with well-known evaluation methods The experimental results show that our classification-based method can accurately evaluate fluency of machine translations.

Listenability Measurement Based on Learners' Transcription Performance.

Language teachers using listening materials on the Internet need to examine the ease of the listening materials (hereafter, listenability) they choose in order to maintain learners’ motivation for listening practice since the listenability of such materials is not controlled, unlike commercially available teaching materials. This study proposes to use a listenability index based on learners’ transcription performance. Transcription performance was determined using the normalized edit distance (hereafter, NED) from a learner’s transcription to a reference sentence. We examined the reliability and validity of NED as a dependent variable for listenability measurement using multiple regression analysis in an experiment comprising 50 learners of English as a foreign language. The results supported the reliability and validity of NED.

Application of a Corpus to Identify Gaps between English Learners and Native Speakers

In order to develop effective computerassisted language teaching systems for learners of English as a foreign language, it is first necessary to identify gaps between learners and native speakers in the four basic linguistic skills (reading, writing, pronunciation, and listening). To identify these gaps, the accuracy and fluency in language use between learners and native speakers should be compared using a learner corpus. However, previous corpora have not included all necessary types of linguistic data. Therefore, in this study, we aimed to design and build a new corpus comprising all types of linguistic data necessary for comparing accuracy and fluency in basic linguistic skills between learners and native speakers.

Measuring Readability for Learners of English as a Foreign Language by Linguistic and Learner Features

The Internet serves as a source of authentic reading material, enabling learners to practice English in real contexts when learning English as a foreign language. An adaptive computer-assisted language learning and teaching system can assist in obtaining authentic materials such as news articles from the Internet. However, to match material level to a learner’s reading proficiency, the system must be equipped with a method to measure proficiency-based readability. Therefore, we developed a method for doing so. With our method, readability is measured through regression analysis using both learner and linguistic features as independent variables. Learner features account for learner reading proficiency, and linguistic features explain lexical, syntactic, and semantic difficulties of sentences. A cross validation test showed that readability measured with our method exhibited higher correlation (r = 0.60) than readability measured only with linguistic features (r = 0.46). A comparison of our method with the method without learner features showed a statistically significant difference. These results suggest the effectiveness of combined learner and linguistic features for measuring reading proficiency-based readability.