Youngjik Lee

The effect of initial weights on premature saturation in back-propagation learning

The critical drawback of the backpropagation learning algorithm is its slow error convergence. The major reason for this is the premature saturation, a phenomenon in which the error of a neural network stays almost constant for some period of time during learning. It is known to be caused by an inappropriate set of initial weights. The probability of incorrectly saturated output nodes at the beginning epoch of learning is derived as a function of the range of initial weights, the number of nodes in each layer, and the maximum slope of the sigmoidal activation function. This is verified by Monte Carlo simulation.<<ETX>>

Sensitivity analysis of single hidden-layer neural networks with threshold functions

An important consideration when applying neural networks to pattern recognition is the sensitivity to weight perturbation or to input errors. In this paper, we analyze the sensitivity of single hidden-layer networks with threshold functions. In a case of weight perturbation or input errors, the probability of inversion error for an output neuron is derived as a function of the trained weights, the input pattern, and the variance of weight perturbation or the bit error probability of the input pattern. The derived results are verified with a simulation of the Madaline recognizing handwritten digits. The result shows that the sensitivity of trained networks is far different from that of networks with random weights.

Phoneme segmentation of continuous speech using multi-layer perceptron

We propose a new method of phoneme segmentation using MLP (multi-layer perceptron). The structure of the proposed segmenter consists of three parts: preprocessor, MLP-based phoneme segmenter, and postprocessor. The preprocessor utilizes a sequence of 44 order feature parameters for each frame of speech, based on the acoustic-phonetic knowledge. The MLP has one hidden layer and an output layer. The feature parameters for four consecutive inter-frame features (176 parameters) are served as input data. The output value decides whether the current frame is a phoneme boundary or not. In postprocessing, we decide the positions of phoneme boundaries using the output of the MLP. We obtained 84% for 5 msec-accuracy and 87% for 15 msec-accuracy with an insertion rate of 9% for open test. By adjusting the threshold value of the MLP output, we achieved higher accuracy. When we decreased the threshold by 0.4, we obtained 5 msec-accuracy of 92% with insertion rate of 3.4% for the insertions that are more than 15 msec apart from phoneme boundaries.

Implementation of the POW (phonetically optimized words) algorithm for speech database

The paper proposes the concept of the POW (phonetically optimized words) set. To collect a speech database, all possible phonological phenomenon should be included. In addition, it is preferable to have the same phonological distribution as the general speech. For this purpose, the authors suggest a new algorithm for selecting a word set which has the properties that (1) it includes all phonological events, (2) it has the minimal number of words, and (3) the phonological similarity between the POW set and the high-frequency word set is maximized. The authors extract the Korean POW set from 50000 high-frequency words out of 3 million text corpus. The POW set is much more similar to the high-frequency word set than the PBW (phonetically balanced words) set with less number of words.

Effect of nonlinear transformations on correlation between weighted sums in multilayer perceptrons

Nonlinear transformation is one of the major obstacles to analyzing the properties of multilayer perceptrons. In this letter, we prove that the correlation coefficient between two jointly Gaussian random variables decreases when each of them is transformed under continuous nonlinear transformations, which can be approximated by piecewise linear functions. When the inputs or the weights of a multilayer perceptron are perturbed randomly, the weighted sums to the hidden neurons are asymptotically jointly Gaussian random variables. Since sigmoidal transformation can be approximated piecewise linearly, the correlations among the weighted sums decrease under sigmoidal transformations. Based on this result, we can say that sigmoidal transformation used as the transfer function of the multilayer perceptron reduces redundancy in the information contents of the hidden neurons.

Synthetic and SNHC audio in MPEG-4

In addition to its sophisticated audio-compression capabilities, MPEG-4 contains extensive functions supporting synthetic sound and the synthetic/natural hybrid coding of sound. We present an overview of the Structured Audio format, which allows efficient transmission and client-side synthesis of music and sound effects. We also provide an overview of the Text-to-Speech Interface, which standardizes a single format for communication with speech synthesizers. Finally, we present an overview of the AudioBIFS portion of the Binary Format for Scene Description, which allows the description of hybrid soundtracks, 3-D audio environments, and interactive audio programming. The tools provided for advanced audio functionality in MPEG-4 are a new and important addition to the world of audio standards.

Focusing and leveling system using position-sensitive detectors for the wafer steppers

An optical focus and leveling system for ETRI KrF excimer laser stepper is developed using position sensitive detectors (PSD) and optical magnification method. This type of detection method showed focusing and leveling accuracies of about +/- 0.1 micrometers and +/- 1.0 arcsec (+/- 0.5 X 10-5 rad) respectively. Also, we confirmed experimentally the autofocus system has +/- 0.15 micrometers signal stability within the controlled temperature range of +/- 0.1 degree(s)C. In this paper, we report the design concepts of the focusing and leveling system and the characteristics of the system parameter applied to ETRI KrF excimer laser stepper.

A conceptual interpretation of spurious memories in the Hopfield-type neural network

It is shown that the spurious memories are represented as logical combinations of the learned memories. By assigning a conceptual interpretation to each learned memory, the spurious memories can be interpreted as novel conceptual knowledge created by the network. It is proposed that the generation of spurious memories be considered a primitive creativity that the simple network exhibits in high-level information processing.<<ETX>>

Semi-automatic construction of Korean-Chinese verb patterns based on translation equivalency

This paper addresses a new method of constructing Korean-Chinese verb patterns from existing patterns. A verb pattern is a subcategorization frame of a predicate extended by translation information. Korean-Chinese verb patterns are invaluable linguistic resources that are not only used for Korean-Chinese transfer but also for Korean parsing. Usually a verb pattern has been either hand-coded by expert lexicographers or extracted automatically from bilingual corpus. In the first case, the dependence on the linguistic intuition of lexicographers may lead to the incompleteness and the inconsistency of a dictionary. In the second case, extracted patterns can be domain-dependent. In this paper, we present a method to construct Korean-Chinese verb patterns semi-automatically from existing Korean-Chinese verb patterns that are manually written by lexicographers.

Toward translating Korean speech into other languages

The paper describes research activities of ETRI in multi lingual spontaneous speech translation. We have developed a Korean to English, Korean to Japanese speech translation system prototype that includes a 5,000 word spontaneous Korean speech recognizer, Korean-English and Korean-Japanese translators, and a Korean speech synthesizer with spontaneous prosody in a travel planning task. We utilize multimedia communication to increase the performance of the spoken language translation system.