Yoshiro Miyata

Optical neural networks using a new radial nonlinear neural layer

Abstract Radially nonlinear neurons are introduced, and back propagation learning for multi-layer networks of these simple hidden units is derived and simulated. The nonlineartransformation performed by a hidden layer of radial units can be represented asa simple multiplication of the summed net input to each neuron by a single valuewhich is only dependent on the total input to the hidden layer. This allows a simpleoptical implementation, in which a single modulator/detector is able to act as anentire hidden layer by multiplexing the neuron net inputs and processed outputs. 1. Introduction A unique advantage of optical systems in computing is the ability of many free space optical beams to cross through the same volume without interacting. This property has been used advantageously for linear transfor-mations, interconnections,1 and has been suggested for simultaneous parallel access to the pixels of an optical memory by multiple processor elements.21 However, this property has not been used in a nonlinear optical systemperforming a logical or neural type of operation. Even the idea of multiple nonlinear logic gates sharing the samehardware is foreign to the notion of point nonlinearities for logic, which of course requires no cross talk betweengates. In this paper we introduce a new type of neural nonlinearity that relies on a radial nonlinear mapping in anN-dimensional vector space, as opposed to pointwise nonlinearities normally performed by a layer of independentneurons. We have found a way to incorporate this new radial nonlinearity into an optical neural network in orderto achieve a fundamental advantage in energy per operation for an optical radial neuron.