Shigeo Sakaue

Reduction of required precision bits for back-propagation applied to pattern recognition

The number of precision bits for operations and data are limited in the hardware implementations of backpropagation (BP). Reduction of rounding error due to this limited precision is crucial in the implementation. The new learning algorithm is based on overestimation of significant error in order to alleviate underflow and omission of weight updating for correctly recognized patterns. While the conventional BP algorithm minimizes the squared error between output signals and supervising data, the new learning algorithm minimizes the weighted error function. In the learning simulation of multifont capital recognition, this algorithm converged recognition accuracy to 100% with only 8-b precision. In addition, the recognition accuracy for characters that did not appear in the training data reached 94.9%. This performance is equivalent to that of a conventional BP with 12-b precision. Moreover, it is found that the performance of the weighted error function is high even when only a small number of hidden neurons is used. Consequently, the algorithm reduces the required amount of weight memory.

Evaluation of nonlinear optimization methods for the learning algorithm of artificial neural networks

This paper describes the implementation of nonlinear optimization methods into the learning of neural networks (NN) and the speed efficiency of four proposed improvements into the backpropagation algorithm. The problems of the backpropagation learning method are pointed out first, and the efficiency of implementing a nonlinear optimization method as a solution to this problems is described. Two nonlinear optimization methods are selected after inspecting several nonlinear methods from the viewpoint of NN learning to avoid the problem of the backpropagation algorithm. These are the linear search method by Davies, Swann, and Campey (DSC), and the conjugate gradient method by Fletcher and Reeves. The NN learning algorithms with these standard methods being implemented are formulated. Moreover, the following four improvements of the nonlinear optimization methods are proposed to shorten the NN learning time: (a) fast forward calculation in linear search by consuming a larger amount of memories; (b) avoiding the trap to local minimum point in an early stage of linear search; (c) applying a linear search method suitable for parallel processing; and (d) switching the gradient direction using the conjugate gradient method. The evaluation results have shown that all methods described here are effective in shortening the learning time.

Solid State Imaging Techniques. Adaptive Gamma Processing of the Video Camera for the Expansion of the Dynamic Range.

We have developed a new signal processing method for expanding the dynamic range of a video camera. A variable and nonlinear gamma characteristic is applied to the input image depending on the distribution of the luminance. We set the gamma characteristic for the back-lit images so as to amplify the luminance of the dark pixels and preserve the contrast of the bright pixels. We have established the decision rule of the gamma characteristic using the learning algorithms of neural networks in order to make the decision rule correspond human vision. The implementation of the gamma decision rule consists of a cascade connection of RAMs, which decreases the required total capacity of RAMs by 1/100 compared with the implementation with a single RAM. The effect of our new method is expand the dynamic range by three times.