Yasuharu Shimeki

Reduction of required precision bits for back-propagation applied to pattern recognition

The number of precision bits for operations and data are limited in the hardware implementations of backpropagation (BP). Reduction of rounding error due to this limited precision is crucial in the implementation. The new learning algorithm is based on overestimation of significant error in order to alleviate underflow and omission of weight updating for correctly recognized patterns. While the conventional BP algorithm minimizes the squared error between output signals and supervising data, the new learning algorithm minimizes the weighted error function. In the learning simulation of multifont capital recognition, this algorithm converged recognition accuracy to 100% with only 8-b precision. In addition, the recognition accuracy for characters that did not appear in the training data reached 94.9%. This performance is equivalent to that of a conventional BP with 12-b precision. Moreover, it is found that the performance of the weighted error function is high even when only a small number of hidden neurons is used. Consequently, the algorithm reduces the required amount of weight memory.

Magnetoresistive thin film head

A magnetoresistive type thin film magnetic head is completely unaffected by external noise and produces only low interference with a recording medium. A magnetoresistive element which constitutes the thin film magnetic head has three terminals constituted by two end terminals and an intermediate terminal. Constant currents which flow in opposite directions are respectively supplied to the two end terminals. A magnetic field of a signal recorded on and reproduced from a single track of the recording medium is applied to the magnetoresistive element, and reproduction outputs of opposite phases appear at the two end terminals. These outputs are differentially amplified by a differential amplifier. Furthermore, the magnetoresistive element is biased by an induced magnetic anisotropy means in a predetermined direction.

Recognition of handwritten numeric characters using neural networks designed on approximate reasoning architecture

We have newly developed a handwritten numeric character recognition system with neural networks based on an approximate reasoning architecture (NARA). Handwritten character recognition is one of the most difficult tasks in an area of pattern recognition because of the variation of handwritten images even in a same category of character. NARA, which consists of a classifier of input data, several sub-neural networks and an integrator of the outputs of sub-neural networks can realize a stable recognition of large variations of handwritten character images, and achieved a correct answer rate of 95.41%, an error rate of 0.20% and a rejection rate of 4.38%.

Adaptive segmentation of quantizer neuron architecture (ASQA)

The authors have previously proposed a multi functional layered network (MFLN) employing a quantizer neuron model and proved that a learning speed of MFLN is the fastest among RCE networks, LVQ3 and multi-layered neural network with backpropagation. The authors also proved that MFLN has very nice supplemental learning performance and can realize adaptive learning or filtering. One of the biggest issues of neural networks is how to design the network structure. In this paper the authors propose an adaptive segmentation of quantizer neuron architecture (ASQA) for answering the above issue and apply them to handwritten character recognition. The networks based on ASQA consist of quantizer neurons which can proliferate themselves and form the optimum network structure for the recognition automatically during training. As a result, there is no need to design the structure of the networks and the average accuracy of the closed and the open test of 27,200 handwritten numeric characters increased to 99.6%. The best tuning of a segmentation threshold of quantizer neurons produced the optimum network size of ASQA.

Segmentation of handwritten Japanese character strings with Hopfield type neural networks

Whereas a character segmentation is an essential pre-process for performing a character recognition, this has been an extremely complicated task for Japanese document recognition. The difficulties of it are due to the irregularities of sizes and disposition of Japanese characters in addition to an existence of separated characters. Thus, we have developed a new segmentation method with a Hopfield type neural networks and applied it to handwritten Japanese character strings. A general constraining conditions for segmentation of Japanese characters is expressed as energy functions in the networks and the networks can perform segmentation of Japanese character strings pliably. Our experimental result showed a probability of correct segmentation of 82.8% in contrast to 75.9% obtained by the conventional method.

Object recognition system using temporal pattern recognition networks with quantizer neuron chip

One of the biggest issues of an object recognition is the recognition with rotation invariance under a fluctuating noisy environment. We developed an object recognition system using temporal pattern recognition network with quantizer neuron chip (QNC) and a /spl phi/-s transformation of shapes and applied them to object recognition. The shape of the object is converted to a series of angles as a function of the circumference of the shape (/spl phi/-s data) and can be treated as a series of temporal patterns. The system consists of a multifunctional layered network(MFLN) with QNC and a layer of neurons with self feedback (self feedback layer). The self feedback layer unifies the temporal recognition results of networks with QNC during a certain period defined by the time constant of self feedback and this function can realize the function of selective attention to certain areas of a series of temporal patterns. As a result, the system realizes rotation invariance in recognition and we obtained 100% recognition accuracy of 50 trials with fluctuating noise taken by CCD camera.

Reduction of necessary precision for the learning of pattern recognition

The authors propose a novel learning algorithm with weighted error function (WEF). They have reduced the necessary precision for the learning of multi-font alpha-numeric recognition to 10-bit fixed point precision using the WEF. The WEF raises the recognition accuracy by more than 25% when the precision of all operations (including multiplication and addition) and the precision of all data (including weights and backpropagation signals) are limited to 10-bit fixed point. This improves the feasibility of analog implementation and lessens the data width of digital implementation. The performance of the WEF is high even with a small number of hidden neurons. This enables the reduction of weight memory. Furthermore, the WEF accelerates the learning and thus refines the adaptability of backpropagation.<<ETX>>

Character recognition system with cooperation of pattern and symbolic processing

A newly developed character recognition method is proposed that can be applied to low quality printed documents. In this method, the cooperation of pattern processing with neural networks and symbolic processing with knowledge of language is adopted. If errors occur at one part, another part detects it and sends the error information to all parts. After successive iterations until no error is detected, a recognition result is obtained. A character recognition of 98.4 percent is obtained with this method. This rate is 2.8 percent higher than the result of a conventional method with no information exchange among processing parts.

Multifunctional layered network with quantizer neurons

The authors propose a multifunctional layered network (MFLN) with a quantizer neuron model and describe the principles of the quantizer neuron and the structure of the network for a character recognition system. Each layer of the MFLN has a specific function defined by the quantizer input of the quantizer neuron, and its learning speed is extremely fast. The authors have applied it to a character recognition system and tested its initial and supplemental learning performance in comparison with three other network models (RCE networks, LVQ3, and a multilayered neural network with back-propagation). For initial learning of ten fonts, the MFLN is fastest, and it is 40 times faster than the multilayered neural network with back-propagation. For supplemental learning with seven further fonts also, the MFLN is the fastest, and it is 600 times faster than the multilayered neural network with back-propagation. The recognition rate for 10 of the fonts learned initially is 97.4% after the MFLN has learned supplementary fonts, and the MFLN displays the lowest degradation of the recognition rate of initially learned fonts.<<ETX>>