A wave-pulse neural network for quasi-quantum coding
AA wave-pulse neural network for quasi-quantum coding
Wen-Zhuo Zhang ∗ AGI Lab, Qusute Ltd., Beijing, P. R. China (Dated: February 11, 2020)We design a physical wave-pulse neural network (WPNN) for both wave and pulse propagation, which givesmore degrees of freedom for neural coding than spike neural networks (SNN). We define the rules and theinformation entropy of this kind of neural network, where the signal speed, arrival time, and the length ofconnections between neurons all become crucial parameters for signal coding. We call it quasi-quantum coding(QQC) since the combination of wave and pulse signals here behaves like a classical mimic of quantum wave-particle duality, and can be studied by borrowing some concepts form quantum mechanics. We present that thequasi-quantum coding can give e ffi cient methods for both sound and image recognitions. We also discuss thepossibility of the wave-pulse neural network and the quasi-quantum coding methods running on it in biologicalbrains where both neural oscillations and action potentials are important to cognition. I. INTRODUCTION
Quantum mechanics is a core theory of modern physicswhich dominates the behavior of microscopic world such asatoms, molecules and elementary particles. Whether quan-tum mechanics play an important role in the brain cognitionis a debate for decades. Penrose guessed that quantum me-chanics may play an role [1], while Tegmark argued that thequantum decoherence time is too short for any quantum infor-mation process in brain [2]. Recently, M. Fisher gave a hy-pothesis that the nuclear spins of Posner molecules in neuralsystem may work as long lifetime qubits and form long-rangequantum entanglements [3]. However, the way such quantuminformation processes a ff ecting real neural functions is stillunknown.In neural science, the neurons and their networks ex-ist in a macroscopic scale which is dominated by classicalphysics rather than quantum physics since all quantum ef-fects in a neuron should vanish due to quantum decoherenceas Tegmark argued [2]. On the other hand, recent studies showthat some quantum entanglement e ff ects can be emulated evenin a pure classical system of superposed waves [4, 5]. There-fore, even if the brain neural networks can present some quan-tum e ff ect, it is more likely some kind of emulation (or mimic)of quantum system by classical wave system rather than a truemicroscopic quantum system.In this paper, we define a kind of physical neural networkwith both wave and pulse propagating between neurons by sixrules, and define its information entropy. It is evoked by thesimultaneous propagation of brain-waves and action potentialpulses in the brain, which are both crucial to brain memoryand cognition [6]. In our neural network, any input signal canbe coded by waves and pulses together and form unique neu-ral connection patterns. The physical parameters such as thesignal speed, the arrival time [7], and the length of connec-tions between neurons become essential here. We choose theword ”quasi-quantum” to call this coding method since 1) it isa classical mimic of the wave-particle duality in quantum me-chanics, with wave and pulse part corresponding to the wave ∗ Electronic address: [email protected] and particle property in quantum mechanics, respectively, and2) we can borrow the language from quantum mechanics todefine a neuron with wave and single pulse passing at a timeas a ”ground state” and a neuron with double pulse meeting ata time as an ”excited state”. This mimic has a similar startingpoint to the de Broglie’s original polit-wave theory [8], whichtreat the wave and particle property independently rather thanunify them as quantum mechanics does.
II. THE WAVE-PULSE NEURAL NETWORK AND ITSINFORMATION ENTROPY
We define the wave-pulse neural network (WPNN) with sixrules: 1) the neural network is made up with sub-networks,and there are neuron-to-neuron connections in every sub-network and between every two sub-networks; 2) each sub-network only allows unique frequency wave signals to prop-agate, while pulse signals can propagate through all connec-tions; 3) every wave signal has a constant amplitude, and ev-ery pulse signal have a constant amplitude too, while the am-plitude of a pulse signal is much larger than the amplitude ofa wave signal; 4) the temporal width of a pulse signal is muchnarrower than any period of a wave signal; 5) a neuron is ex-cited when two pulse signals from di ff erent connections arriveat it simultaneously; 6) all signals propagate at a same speed.Fig. 1 gives the schematic connections of the wave-pulseneural network. We call the i th sub-network have a mode of ω i , which means that it only allow the wave signal with fre-quency ω i to propagate. The i th sub-network have M i neurons( M i >> j th neuron in it has C i j connections to otherneurons within the sub-network.For the i th sub-network, we can define an information en-tropy H i = M i − M i (cid:88) j = C i j log M i − C i j , (1)where the information entropy equals to zero if C i j = C i j = H i = a r X i v : . [ q - b i o . N C ] F e b FIG. 1: The wave-pulse neural network is made up with sub-networks and each two sub-networks have neuron-to-neuron connec-tion between them. The i th sub-network only allow the wave signalwith frequency ω i to propagate, while all connections in the networkallow pulse signals to propagate. Wave and pulse signals can overlaptogether. The duration of a pulse signal is much narrower than a pe-riod of a wave signal, while the amplitude of a pulse signal is muchlarger than it of a wave signal M i − (cid:80) M i j = log M i − <<
1. At the maximum network con-nection, each neuron connects M i − H i = N sub-networks and the i th sub-network has C i connections to othersub-networks (neuron-to-neuron), then we can define the totalinformation entropy H = N − N (cid:88) i = H i C i log N − C i = N − N (cid:88) i = ( C i log N − C i )( 1 M i − M i (cid:88) j = C i j log M i − C i j ) , (2)where the information entropy equals to zero if C i = C i = < H i << H = N − (cid:80) Ni = H i log M i − << N − H i = H = H = H =
1, a signal in a neuronhas maximum choices to propagate. Therefore, it can be con-sidered as a generation of Shannon’s information entropy that measures the uncertainty of bits.
III. QUASI-QUANTUM CODING METHOD ON THENETWORK
Since we define a wave-pulse neural network, a wave sig-nal can propagate though all the connections in a sub-networkwhile a pulse signal can propagate though all the connectionsin the whole network. Then which way is adopted for cod-ing depends on whether it can excite a neuron. According tothe rule 5 in section II, two pulse signals form di ff erent con-nections should arrive at a neuron simultaneously to excite it.We define it as an quasi-quantum excitation | ω > with a fre-quency ω from the local wave signal and a particle number n = i th frequency component of an analog sig-nal can be code into a quasi-quantum signal as M i = A w cos( k i x + ω i t ) + N (cid:88) n = A p p n ( t n ) , (3)where A w is the constant amplitude of a wave signal and A p isthe constant amplitude of a pulse signal as defined by rule 3in section II. k i is the wavenumber and ω i is the frequency ofthe wave signal in i th sub-network which are correspondingto the frequency of the signal’s i th frequency component. t n isa series of discrete moments where the signal’s i th frequencycomponent appears, and p n is the number of pulse signals at t n which is corresponding to the amplitude of the signal’s i thfrequency component at t n . Then any analog signal can becode into a series of quasi-quantum signals with decomposingits frequency components.Here a quasi-quantum signal acts like a vacuum state | ω > in its sub-network, and every two pulse signals meeting eachother act like a particle creation operator a † . When two pulsesignals meet in a neuron, a quasi-quantum excitation a † | ω > = | ω > appears and recorded, and the connections between anytwo neurons with quasi-quantum excitations | ω > are alsorecorded. Then an analog signal can be record by a uniqueconnection pattern of neurons with quasi-quantum excitations | ω > over all frequency components it has.Technically, all the quasi-quantum excitations | ω > andtheir time order can be recorded by a time-vs-frequency ma-trix which also imply the connection relations between theneurons with quasi-quantum excitations. Such wave-pulseneural network for quasi-quantum coding is di ff erent to anyartificial neural networks in current computer science dueto that the arrival time of pulses depends on the connectionlengths between neurons and the speed of quasi-quantum sig-nals, which are both physical parameters rather than virtualconnections in computer science. In next two sections, wegive two examples of the quasi-quantum coding method forboth sounds and images, respectively. FIG. 2: Quasi-quantum coding method for simple sound signals.Solid lines represent to sub-networks, and dot lines represent to theconnections between sub-networks for coding. (a) Connection pat-tern of neurons for coding a sound signal with four frequency com-ponents ( ω , ω , ω , ω ) appear in an entering time sequence ( t , t , t , t ); (b) Connection pattern of neurons for coding a sound signalwith for frequency components ( ω , ω , ω , ω ) appear in an enter-ing time sequence ( t , t , t , t ). IV. QUASI-QUANTUM CODING FOR SOUNDS
Sound wave is a mechanic wave signal with frequencies andamplitudes distributing over time. In order to code a soundwave, we can decompose it into its frequency components andcode them with wave signals in di ff erent mode sub-networkrespectively, while we code the amplitude of each frequencycomponent with the number of separated pulse signals at themoment when the frequency component appears.Since a wave signal can travel only in its sub-network butpulse signals can travel along all connections, as rule 2 in sec-tion II, the connections between sub-network only have pulsesignals. Besides, all signals travel at a same constant speed,as rule 6 in section II. Therefore, only the arrival time of pulsesignals for the coding is necessary. Fig.2 gives a simple ex-ample of two pulses’ meeting form two directions. Accordingto rule 5 in section II, the neuron where the two pulse meet isa quasi-quantum excitation | ω > . If the time interval of thetwo pulse signals is ∆ t , the length di ff erence between the twoconnection (start form the first neuron of each sub-network) is d = v ∆ t , where v is the constant speed of all signals.For a simple sound signal with four frequency componentswhere each frequency component appears at a di ff erent time, we can code it with four frequency ( ω , ω , ω , ω ) at a timeorder ( t , t , t , t ) that entering the network. In Fig.2(a),a quasi-quantum signal travels in a sub-network ω i with adistance a i and its pulse signals also travels from the sub-network ω i to the sub-network ω j with a distance b i j . Whenthe pulse form ω meets another pulse in ω , the neuronwhere they meet is excited and the distances follows a relation a + b − a = v ( t − t ). Next, when the pulse form ω meetsanother pulse in ω , the relation is a + b − a = v ( t − t ),and when the pulse form ω meets another pulse in ω , therelation is a + b − a = v ( t − t ). In another case, ifwe code a signal with four frequency ( ω , ω , ω , ω ) at aentering time sequence ( t , t , t , t ), the relations become a + b − a = v ( t − t ), a + b − a = v ( t − t ), and a + b − a = v ( t − t ), as Fig.2(b) shows.This quasi-quantum coding method gives a unique networkconnection pattern of all excited neurons for a sound signalin an irreversible time order. When we record all the excitedneurons (as well as the non-excited neuron in the first sub-network that connected to the first excited neuron), we can geta time-vs-frequency matrix, where the transverse direction istime order (from left to right) and the longitudinal directionare frequencies. For the simple example of Fig.2, we can gettwo matrixes p p p
00 0 0 p & p p p p . (4)Here p i is the number of quasi-quantum excitations in the ω i sub-network which is corresponding to the amplitude of thesound signal’s frequency component (coded by ω i ). If thetwo sound signals mix together, the mixed matrix from quasi-quantum coding would be p p p + p p + p p p . (5)All such matrixes have a translation invariance for both fre-quency and entering time of quasi-quantum signals. It meanswe can get a matrix for a sound signal ( ω + ω , ω + ω , ω + ω , ω + ω , · · · ) at time order ( t + t , t + t , t + t , t + t , · · · ) which is same to the matrix for the sound signal( ω , ω , ω , ω , · · · ) at time order ( t , t , t , t , · · · ). Theo ff set frequency ” ω ” and o ff set time ” t ” are free to choose.So it makes the quasi-quantum coding method has symmetriesof frequency translation, which is useful for cognising soundswith same syllables but di ff erent frequencies (pitch). V. QUASI-QUANTUM CODING FOR IMAGES
The quasi-quantum coding method can be applied to codeimages in a similar way to the coding of sounds. Since an im-age is a two dimensional spatial distribution of data, in orderto code it into quasi-quantum signals on a wave-pulse neuralnetwork, we need to set a protocol. In digital computers, mostof the images are coded line by line in order to transform two-dimensional spatial data into one-dimensional data over time.However, in our quasi-quantum coding method, we can notdistribute every pixel a frequency since the pixel number of aimage is usually large.Inspired by the active scan ability of animal eyes (saccade)[9], we can use four individual wave-pulse neural networksto acquire signals form scanning. We define NL as the net-work that receives and codes the signals form the left-to-rightscan ” → ”, NR as the network that receives and codes the sig-nals form the right-to-left scan ” ← ”, NT as the network thatreceives and codes the signals form the top-to-bottom scan” ↓ ”, and NB as the network that receives and codes the sig-nals form the bottom-to-top scan ” ↑ ”. Every network hassub-networks with di ff erent frequencies, and each frequencyis corresponding to a unique row of pixels in NL and NR , or aunique column of pixels in NT and NB .With this protocol, we can code any symbol by the quasi-quantum coding method on the four wave-pulse neural net-works, and each network works in a same way to the coding ofsounds where a frequency-vs-time matrix is finally recorded.For example, we can code a ”T” type symbol into quasi-quantum signals and record four frequency-vs-time matrixesas p . . . p . . . ... ... . . . & p . . . p . . . ... ... . . . & . . . ... ... . . . p . . . p . . . ... ... . . . & . . . ... ... . . . p . . . p . . . ... ... . . . . (6)where the matrixes lie in a NL & NR & NT & NB way. p is thenumber of pulses on every neuron with quasi-quantum excita-tions.For a ” + ” type symbol, we can record four same frequency-vs-time matrixes as . . . ... ... . . . p . . . p . . . ... ... . . . & . . . ... ... . . . p . . . p . . . ... ... . . . & . . . ... ... . . . p . . . p . . . ... ... . . . & . . . ... ... . . . p . . . p . . . ... ... . . . , (7)For a ”X” type symbol, we can record four same frequency- vs-time matrixes as p . . . p p ... p ... ... . . . ... ... p ... p p . . . p & p . . . p p ... p ... ... . . . ... ... p ... p p . . . p & p . . . p p ... p ... ... . . . ... ... p ... p p . . . p & p . . . p p ... p ... ... . . . ... ... p ... p p . . . p . (8)For any curve, we can record four frequency-vs-time ma-trixes with p distributes as a curve in each matrix. For exam-ple, we can get four same frequency-vs-time matrixes for an”O” type symbol, and in each matrix, the nonzero elements(pulse number) distribute like a circle (or a ellipse, dependson the spatial and temporal resolutions).In our protocol, an image with 1000 × ,
000 sub-networks for this image.In order to make this coding method work, all the pixels of animage should be acquired parallel, which mean that we needto set a CCD or a CMOS camera to a parallel mode and leadsignals of all pixels parallel into our physical networks.For scale-invariant image cognition, our quasi-quantumcoding method can use a camera with auto-focus mode, whichcan make similar images with di ff erent sizes to have similarinputs of pixels and finally get similar matrixes. We can alsoselect a range of interest of each image to include the simi-lar parts over all images, unify the pixels of range of interests(usually reduce the pixels proportionally to the lowest ones),and use quasi-quantum coding method to output similar ma-trixes. VI. CONCLUSION AND DISCUSSIONS
We design a wave-pulse neural network with six rules andinformation entropy. The information entropy is proportionalto the connection complexity of the neural network, thus lessconnections mean less information entropy. As we know inneural science, a brain has more synapse connections betweenneurons in baby phase, and learning is a process of reduc-ing connections between neurons [10]. So if the animal brainis (or partly is) a biological wave-pulse neural network, thelearning process in a brain is a process of reducing informa-tion entropy.We also define the quasi-quantum coding methods forsounds and images that work on the wave-pulse neural net-work. For coding sounds, we use a simplified quasi-quantumcoding method where each frequency have a unique sub-network to propagate. A typical hearing range of a human isup to 20kHz. If we set the frequency resolution as 1Hz, a com-mon sound signal may requires thousands of sub-networks tocode. In order to code sound signals for a much wider fre-quency range with less sub-networks, we can set a binary acti-vation of N sub-networks. For example, sixteen sub-networkshave 2 − = ,
535 binary activation modes (with 1 meansactive and 0 means non-active for a sub-network), which cancode sound signals up to 60kHz with 1Hz resolution.For coding images, we use a quasi-quantum coding methodwith scanning a image for four time (” → ”, ” ← ”, ” ↓ ” and ” ↑ ”),and every scan is similar to the coding method sounds whereeach row or column of pixels is corresponding to a uniquesub-network. When human eyes receive images, they makesaccade that scanning the features of an image rapidly to getinformation [9]. The scan traces of the saccade may oblique oreven curved. In our quasi-quantum coding method, we havefour rectangularly scan ways. Technically, we can superim-pose any two of the four scan ways and change the scan speed of them individually to get any trace we want.Since a signal can be coded into a unique connection patternof neurons with quasi-quantum excitations | ω > in a wave-pulse neural network, it is satisfied with the memory mechan-ics of a biological neural network where a signal is stored bystrengthening a unique connection among some neurons [10].Besides, the quasi-quantum coding method for both sound andany scan of image are not time reversal. It is also satisfied withthe fact that any memory in an animal brain is time-ordered[10].Therefore, it is interesting to test whether the wave-pulseneural network and the quasi-quantum coding methods run-ning on it are involved in animal brain. In another way, arti-ficial wave-pulse neural network could be built physically, orsimulated on computers at first. More quasi-quantum codingmethods besides coding sounds and images could be devel-oped and run on these wave-pulse neural networks in order tobenefit the development of artificial general intelligence. [1] R. Penrose, The Emperor’s New Mind, Oxford Univ. Press(1989); R. Penrose, in The Large, the Small and the HumanMind, Cambridge Univ. Press (1997).[2] M. Tegmark, Phys. Rev. E , 4194 (2000).[3] M. P. A. Fisher, arXiv:1508.05929.[4] B. R. La Cour and G. E. Ott, New J. Phys. ,611 (2015). [6] J. Fell and N. Axmacher, Nat. Rev. Neurosci. , 105 (2011).[7] S. J. Thorpe, Parallel processing in neural systems and comput-ers 91-94 (1990).[8] L. de Broglie, Found. Phys. , 5 (1970).[9] K. R. Gegenfurtner, Perception45